1
|
Hwang M, Jiang WC, Chen YJ, Hwang KS, Tseng YC. An Efficient Unified Approach Using Demonstrations for Inverse Reinforcement Learning. IEEE Trans Cogn Dev Syst 2021. [DOI: 10.1109/tcds.2019.2957831] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
2
|
Singh B, Kumar R, Singh VP. Reinforcement learning in robotic applications: a comprehensive survey. Artif Intell Rev 2021. [DOI: 10.1007/s10462-021-09997-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
3
|
Heng W, Pang G, Xu F, Huang X, Pang Z, Yang G. Flexible Insole Sensors with Stably Connected Electrodes for Gait Phase Detection. SENSORS 2019; 19:s19235197. [PMID: 31783618 PMCID: PMC6928953 DOI: 10.3390/s19235197] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/18/2019] [Revised: 11/19/2019] [Accepted: 11/21/2019] [Indexed: 11/16/2022]
Abstract
Gait analysis is an important assessment tool for analyzing vital signals collected from individuals and for providing physical information of the human body, and it is emerging in a diverse range of application scenarios, such as disease diagnosis, fall prevention, rehabilitation, and human–robot interaction. Herein, a kind of surface processed conductive rubber was designed and investigated to develop a pressure-sensitive insole to monitor planar pressure in a real-time manner. Due to a novel surface processing method, the pressure sensor was characterized by stable contact resistance, simple manufacturing, and high mechanical durability. In the experiments, it was demonstrated that the developed pressure sensors were easily assembled with the inkjet-printed electrodes and a flexible substrate as a pressure-sensitive insole while maintaining good sensing performance. Moreover, resistive signals were wirelessly transmitted to computers in real time. By analyzing sampled resistive data combined with the gait information monitored by a visual-based reference system based on machine learning method (k-Nearest Neighbor algorithm), the corresponding relationship between plantar pressure distribution and lower limb joint angles was obtained. Finally, the experimental validation of the ability to accurately divide gait into several phases was conducted, illustrating the potential application of the developed device in healthcare and robotics.
Collapse
Affiliation(s)
- Wenzheng Heng
- State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China; (W.H.); (G.P.)
| | - Gaoyang Pang
- State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China; (W.H.); (G.P.)
| | - Feihong Xu
- Institute of Mechanical Manufacturing Technology, China Academy of Engineering Physics, Mianyang 621900, China;
| | - Xiaoyan Huang
- College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China;
| | - Zhibo Pang
- ABB Corporate Research Sweden, 72178 Vasteras, Sweden;
| | - Geng Yang
- State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China; (W.H.); (G.P.)
- Correspondence:
| |
Collapse
|
4
|
Liang Y, Xiao M, Tang X, Ge Y, Wang X. A Q-learning based method of optimal fault diagnostic policy with imperfect tests. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2019. [DOI: 10.3233/jifs-181799] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Affiliation(s)
- Yajun Liang
- Aeronautics Engineering College, Air Force Engineering University, Xi’an, China
| | - Mingqing Xiao
- Aeronautics Engineering College, Air Force Engineering University, Xi’an, China
| | - Xilang Tang
- Aeronautics Engineering College, Air Force Engineering University, Xi’an, China
| | - Yawei Ge
- Aeronautics Engineering College, Air Force Engineering University, Xi’an, China
| | - Xiaofei Wang
- Aeronautics Engineering College, Air Force Engineering University, Xi’an, China
| |
Collapse
|
5
|
Constrained FC 4D MITPs for Damageable Substitutable and Complementary Items in Rough Environments. MATHEMATICS 2019. [DOI: 10.3390/math7030281] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Very often items that are substitutable and complementary to each other are sent from suppliers to retailers for business. In this paper, for these types of items, fixed charge (FC) four-dimensional (4D) multi-item transportation problems (MITPs) are formulated with both space and budget constraints under crisp and rough environments. These items are damageable/breakable. The rates of damageability of the items depend on the quantity transported and the distance of travel i.e., path. A fixed charge is applied to each of the routes (independent of items). There are some depots/warehouses (origins) from which the items are transported to the sales counters (destinations) through different conveyances and routes. In proposed FC 4D-MITP models, per unit selling prices, per unit purchasing prices, per unit transportation expenditures, fixed charges, availabilities at the sources, demands at the destinations, conveyance capacities, total available space and budget are expressed by rough intervals, where the transported items are substitutable and complementary in nature. In this business, the demands for the items at the destinations are directly related to their substitutability and complementary natures and prices. The suggested rough model is converted into a deterministic one using lower and upper approximation intervals following Hamzehee et al. as well as Expected Value Techniques. The converted model is optimized through the Generalized Reduced Gradient (GRG) techniques using LINGO 14 software . Finally, numerical examples are presented to illustrate the preciseness of the proposed model. As particular cases, different models such as 2D, 3D FCMITPs for two substitute items, one item with its complement and two non substitute non complementary items are derived and results are presented.
Collapse
|
6
|
Huan TT, Van Kien C, Anh HPH, Nam NT. Adaptive gait generation for humanoid robot using evolutionary neural model optimized with modified differential evolution technique. Neurocomputing 2018. [DOI: 10.1016/j.neucom.2018.08.074] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
|
7
|
Bai J, Song A, Xu B, Nie J, Li H. A Novel Human-Robot Cooperative Method for Upper Extremity Rehabilitation. Int J Soc Robot 2017. [DOI: 10.1007/s12369-016-0393-4] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
8
|
Parameterized gait pattern generator based on linear inverted
pendulum model with natural ZMP references. KNOWL ENG REV 2016. [DOI: 10.1017/s0269888916000138] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
AbstractThis paper presents a parameterized gait generator based on linear inverted
pendulum model (LIPM) theory, which allows users to generate a natural gait
pattern with desired step sizes. Five types of zero moment point (ZMP)
components are proposed for formulating a natural ZMP reference, where ZMP moves
continuously during single support phases instead of staying at a fixed point in
the sagittal and lateral plane. The corresponding center of mass (CoM)
trajectories for these components are derived by LIPM theory. To generate a
parameterized gait pattern with user-defined parameters, a gait planning
algorithm is proposed, which determines related coefficients and boundary
conditions of the CoM trajectory for each step. The proposed parameterized gait
generator also provides a concept for users to generate gait patterns with
self-defined ZMP references by using different components. Finally, the
feasibility of the proposed method is validated by the experimental results with
a teen-sized humanoid robot, David, which won first place in the sprint event at
the 20th Federation of International Robot-soccer Association (FIRA) RoboWorld
Cup.
Collapse
|
9
|
Abstract
SUMMARYThis paper designs a biped robot to perform appropriate walking exercises according to the terrain, which then walks stably on a flat environment. The concept of fuzzy logic is combined with the Linear Quadratic Regulator (LQR) controller theory to design the best method to allow the biped robot system to have a balanced and stable gait. Traditional controllers are designed using mathematical models of physical systems, but a fuzzy controller is a physical system that uses an inexact mathematical model, which involves sets and membership functions. Fuzzy controllers use fuzzification, fuzzy control rules, and defuzzification. The method and theory of control: A stable gait for robots is achieved using inverse kinematics, fuzzy concepts, the LQR controller theory, path design, and the characteristics of a dynamic equation. It is then simulated using mathematical tools to prove that the system eliminates swinging by biped robots without fuzzy control knowing beforehand the dynamic model the system is using. Proportional-Integral-Differential control achieves a stable gait design in a flat environment.
Collapse
|
10
|
Jennings AL, Ordóñez R. Unbounded Motion Optimization by Developmental Learning. IEEE TRANSACTIONS ON CYBERNETICS 2013; 43:1178-1188. [PMID: 26502428 DOI: 10.1109/tsmcb.2012.2226026] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
An algorithm is presented for autonomous motion development with unbounded waveform resolution. Rather than a single optimization in a very large space, memory is built to support incremental improvements; therefore, complexity is balanced by experience. Analogously, human development manages complexity by limiting it during initial learning stages. Motions are represented by cubic spline interpolation; therefore, the development technique applies broadly to function optimization. Adding a node to the splines allows all previous memory samples to transfer to the higher dimension space exactly. The memory-based model, which is a locally weighted regression (LWR), predicts the expected outcome for a motion and provides gradient information for optimizing the motion. Results are compared against bootstrapping a direct optimization (DO) on a mathematical problem. Additionally, the method has been implemented to learn voltage profiles with the lowest peak current for starting a motor. This method shows practical accuracy and scalability.
Collapse
|
11
|
Nassour J, Hugel V, Ben Ouezdou F, Cheng G. Qualitative adaptive reward learning with success failure maps: applied to humanoid robot walking. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2013; 24:81-93. [PMID: 24808209 DOI: 10.1109/tnnls.2012.2224370] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
In the human brain, rewards are encoded in a flexible and adaptive way after each novel stimulus. Neurons of the orbitofrontal cortex are the key reward structure of the brain. Neurobiological studies show that the anterior cingulate cortex of the brain is primarily responsible for avoiding repeated mistakes. According to vigilance threshold, which denotes the tolerance to risks, we can differentiate between a learning mechanism that takes risks and one that averts risks. The tolerance to risk plays an important role in such a learning mechanism. Results have shown the differences in learning capacity between risk-taking and risk-avert behaviors. These neurological properties provide promising inspirations for robot learning based on rewards. In this paper, we propose a learning mechanism that is able to learn from negative and positive feedback with reward coding adaptively. It is composed of two phases: evaluation and decision making. In the evaluation phase, we use a Kohonen self-organizing map technique to represent success and failure. Decision making is based on an early warning mechanism that enables avoiding repeating past mistakes. The behavior to risk is modulated in order to gain experiences for success and for failure. Success map is learned with adaptive reward that qualifies the learned task in order to optimize the efficiency. Our approach is presented with an implementation on the NAO humanoid robot, controlled by a bioinspired neural controller based on a central pattern generator. The learning system adapts the oscillation frequency and the motor neuron gain in pitch and roll in order to walk on flat and sloped terrain, and to switch between them.
Collapse
|