Efficient Reinforcement Learning via Probabilistic Trajectory Optimization

A decoupled Bayesian method for snake robot control in unstructured environment

This paper presents a method which avoids the common practice of using a complex coupled snake robot model and performing kinematic analysis for control in cluttered environments. Instead, we introduce a completely decoupled dynamical Bayesian formulation with respect to interacted snake robot links and environmental objects, which requires much lower complexity for efficient and robust control. When a snake robot does not interact with obstacles, it runs by a simple serpenoid controller. However, when it exhibits interaction with environments, defined as close proximity or collision with targets and/or obstacles, we extend the conventional Bayesian framework by modeling such interactions in terms of stimuli. The proposed 'multi-neural-stimulus function' represents the cumulative effect of both external environmental influences and internal constraints of the snake robot. It implicitly handles the 'unexpected collision' problem and thus solve the difficult data association and shape adjustment problems for snake robot control in an innovative way. Preliminary experimental results have demonstrated promising performance of the proposed method comparing with the state-of-the-art.

A Survey on Learning-Based Model Predictive Control: Toward Path Tracking Control of Mobile Platforms

The learning-based model predictive control (LB-MPC) is an effective and critical method to solve the path tracking problem in mobile platforms under uncertain disturbances. It is well known that the machine learning (ML) methods use the historical and real-time measurement data to build data-driven prediction models. The model predictive control (MPC) provides an integrated solution for control systems with interactive variables, complex dynamics, and various constraints. The LB-MPC combines the advantages of ML and MPC. In this work, the LB-MPC technique is summarized, and the application of path tracking control in mobile platforms is discussed by considering three aspects, namely, learning and optimizing the prediction model, the controller design, and the controller output under uncertain disturbances. Furthermore, some research challenges faced by LB-MPC for path tracking control in mobile platforms are discussed.

Approximate Policy-Based Accelerated Deep Reinforcement Learning

In recent years, the deep reinforcement learning (DRL) algorithms have been developed rapidly and have achieved excellent performance in many challenging tasks. However, due to the complexity of network structure and a large amount of network parameters, the training of deep network is time-consuming, and consequently, the learning efficiency of DRL is limited. In this paper, aiming to speed up the learning process of DRL agent, we propose a novel approximate policy-based accelerated (APA) algorithm from the viewpoint of the error analysis of approximate policy iteration reinforcement learning algorithms. The proposed APA is proven to be convergent even with a more aggressive learning rate, making the DRL agent have a faster learning speed. Furthermore, to combine the accelerated algorithm with deep Q-network (DQN), Double DQN and deep deterministic policy gradient (DDPG), we proposed three novel DRL algorithms: APA-DQN, APA-Double DQN, and APA-DDPG, which demonstrates the adaptability of the accelerated algorithm with DRL algorithms. We have tested the proposed algorithms on both discrete-action and continuous-action tasks. Their superior performance demonstrates their great potential in the practical applications.

Seizure Reduction using Model Predictive Control

This study presents a model predictive control approach for seizure reduction in a computational model of epilepsy. The differential dynamic programming (DDP) algorithm is implemented in a model predictive fashion to optimize a controller for suppressing seizures in a chaotic oscillator model. The chaotic oscillator model uses proportional-integral (PI) controller to represent the internal control mechanism that maintains stable neural activity in a healthy brain. In the pathological case, the gains of this PI controller are reduced, preventing sufficient feedback to suppress correlation increase between normal and pathological brain dynamics. This increase in correlation leads to synchronization of oscillator dynamics leading to the destabilization of neural activity and epileptic behavior. The pathological case of the chaotic oscillator model is formulated as an optimal control problem, which we solve using the dynamic programming principle. We propose using model predictive control with differential dynamic programming optimization as a possible method for controlling epileptic seizures in known models of epilepsy.