Six-DOF Spacecraft Optimal Trajectory Planning and Real-Time Attitude Control: A Deep Neural Network-Based Approach

Design and Experimental Validation of Deep Reinforcement Learning-Based Fast Trajectory Planning and Control for Mobile Robot in Unknown Environment

This article is concerned with the problem of planning optimal maneuver trajectories and guiding the mobile robot toward target positions in uncertain environments for exploration purposes. A hierarchical deep learning-based control framework is proposed which consists of an upper level motion planning layer and a lower level waypoint tracking layer. In the motion planning phase, a recurrent deep neural network (RDNN)-based algorithm is adopted to predict the optimal maneuver profiles for the mobile robot. This approach is built upon a recently proposed idea of using deep neural networks (DNNs) to approximate the optimal motion trajectories, which has been validated that a fast approximation performance can be achieved. To further enhance the network prediction performance, a recurrent network model capable of fully exploiting the inherent relationship between preoptimized system state and control pairs is advocated. In the lower level, a deep reinforcement learning (DRL)-based collision-free control algorithm is established to achieve the waypoint tracking task in an uncertain environment (e.g., the existence of unexpected obstacles). Since this approach allows the control policy to directly learn from human demonstration data, the time required by the training process can be significantly reduced. Moreover, a noisy prioritized experience replay (PER) algorithm is proposed to improve the exploring rate of control policy. The effectiveness of applying the proposed deep learning-based control is validated by executing a number of simulation and experimental case studies. The simulation result shows that the proposed DRL method outperforms the vanilla PER algorithm in terms of training speed. Experimental videos are also uploaded, and the corresponding results confirm that the proposed strategy is able to fulfill the autonomous exploration mission with improved motion planning performance, enhanced collision avoidance ability, and less training time.

Distributed Neural Networks Training for Robotic Manipulation With Consensus Algorithm

In this article, we propose an algorithm that combines actor-critic-based off-policy method with consensus-based distributed training to deal with multiagent deep reinforcement learning problems. Specifically, convergence analysis of a consensus algorithm for a type of nonlinear system with a Lyapunov method is developed, and we use this result to analyze the convergence properties of the actor training parameters and the critic training parameters in our algorithm. Through the convergence analysis, it can be verified that all agents will converge to the same optimal model as the training time goes to infinity. To validate the implementation of our algorithm, a multiagent training framework is proposed to train each Universal Robot 5 (UR5) robot arm to reach the random target position. Finally, experiments are provided to demonstrate the effectiveness and feasibility of the proposed algorithm.

Autonomous localized path planning algorithm for UAVs based on TD3 strategy

Unmanned Aerial Vehicles are useful tools for many applications. However, autonomous path planning for Unmanned Aerial Vehicles in unfamiliar environments is a challenging problem when facing a series of problems such as poor consistency, high influence by the native controller of the Unmanned Aerial Vehicles. In this paper, we investigate reinforcement learning-based autonomous local path planning methods for Unmanned Aerial Vehicles with high autonomous decision-making capability and locally high portability. We propose an autonomous local path planning algorithm based on the TD3 strategy to solve the problem of local obstacle avoidance and path planning in unfamiliar environments using autonomous decision-making of Unmanned Aerial Vehicles. The simulation results on Gazebo show that our method can effectively realize the autonomous local path planning task for Unmanned Aerial Vehicles, the success rate of path planning with our method can reach 93% under the interference of no obstacles, and 92% in the environment with obstacles. Finally, our method can be used for autonomous path planning of Unmanned Aerial Vehicles in unfamiliar environments.

Parameter Space Optimization for Robust Controller Synthesis With Structured Feedback Gain

Various optimal control and system designs involve searching for a feedback gain matrix with structural constraints. As an alternative solution, the parameter space methods map the constraints from the state space to another extended state-input space, in which an equivalent optimization problem is solved. However, to further extend its applications, there are still some issues need to be addressed, such as the limited type of structural constraints, the marginally stable solutions, and the low computation efficiency. In this article, we aim to make this method applicable to a class of structural constraints for some elements in the gain matrix being zero or with intrarow and intracolumn constraints. To address such structured control problem, we propose a procedure to transform the original system to an extended system with the decentralized feedback matrix. From here, the mapping rules to the parameter space are given for the decentralized feedback matrix with both intrarow and intracolumn constraints. To avoid oscillatory closed-loop dynamics, we include the closed-loop dominant pole constraints during optimization. In addition, to improve the computation efficiency during optimization, we revise the cutting plane logic, which allows adding multiple linear constraints within a single iteration. Simulation examples demonstrate the effectiveness of the proposed method.

Composite Observer-Based Optimal Attitude-Tracking Control With Reinforcement Learning for Hypersonic Vehicles

This article proposes an observer-based reinforcement learning (RL) control approach to address the optimal attitude-tracking problem and application for hypersonic vehicles in the reentry phase. Due to the unknown uncertainty and nonlinearity caused by parameter perturbation and external disturbance, accurate model information of hypersonic vehicles in the reentry phase is generally unavailable. For this reason, a novel synchronous estimation is proposed to construct a composite observer for hypersonic vehicles, which consists of a neural-network (NN)-based Luenberger-type observer and a synchronous disturbance observer. This solves the identification problem of nonlinear dynamics in the reference control and realizes the estimation of the system state when unknown nonlinear dynamics and unknown disturbance exist at the same time. By synthesizing the information from the composite observer, an RL tracking controller is developed to solve the optimal attitude-tracking control problem. To improve the convergence performance of critic network weights, concurrent learning is employed to replace the traditional persistent excitation condition with a historical experience replay manner. In addition, this article proves that the weight estimation error is bounded when the learning rate satisfies the given sufficient condition. Finally, the numerical simulation demonstrates the effectiveness and superiority of the proposed approaches to attitude-tracking control systems for hypersonic vehicles.

Evolving deep convolutional neutral network by hybrid sine-cosine and extreme learning machine for real-time COVID19 diagnosis from X-ray images

The COVID19 pandemic globally and significantly has affected the life and health of many communities. The early detection of infected patients is effective in fighting COVID19. Using radiology (X-Ray) images is, perhaps, the fastest way to diagnose the patients. Thereby, deep Convolutional Neural Networks (CNNs) can be considered as applicable tools to diagnose COVID19 positive cases. Due to the complicated architecture of a deep CNN, its real-time training and testing become a challenging problem. This paper proposes using the Extreme Learning Machine (ELM) instead of the last fully connected layer to address this deficiency. However, the parameters' stochastic tuning of ELM's supervised section causes the final model unreliability. Therefore, to cope with this problem and maintain network reliability, the sine-cosine algorithm was utilized to tune the ELM's parameters. The designed network is then benchmarked on the COVID-Xray-5k dataset, and the results are verified by a comparative study with canonical deep CNN, ELM optimized by cuckoo search, ELM optimized by genetic algorithm, and ELM optimized by whale optimization algorithm. The proposed approach outperforms comparative benchmarks with a final accuracy of 98.83% on the COVID-Xray-5k dataset, leading to a relative error reduction of 2.33% compared to a canonical deep CNN. Even more critical, the designed network's training time is only 0.9421 ms and the overall detection test time for 3100 images is 2.721 s.

Learning to Optimize: Reference Vector Reinforcement Learning Adaption to Constrained Many-Objective Optimization of Industrial Copper Burdening System

The performance of decomposition-based algorithms is sensitive to the Pareto front shapes since their reference vectors preset in advance are not always adaptable to various problem characteristics with no a priori knowledge. For this issue, this article proposes an adaptive reference vector reinforcement learning (RVRL) approach to decomposition-based algorithms for industrial copper burdening optimization. The proposed approach involves two main operations, that is: 1) a reinforcement learning (RL) operation and 2) a reference point sampling operation. Given the fact that the states of reference vectors interact with the landscape environment (quite often), the RL operation treats the reference vector adaption process as an RL task, where each reference vector learns from the environmental feedback and selects optimal actions for gradually fitting the problem characteristics. Accordingly, the reference point sampling operation uses estimation-of-distribution learning models to sample new reference points. Finally, the resultant algorithm is applied to handle the proposed industrial copper burdening problem. For this problem, an adaptive penalty function and a soft constraint-based relaxing approach are used to handle complex constraints. Experimental results on both benchmark problems and real-world instances verify the competitiveness and effectiveness of the proposed algorithm.

Event-based optimal guidance laws design for missile-target interception systems using fuzzy dynamic programming approach

In this paper, the guidance system with unknown dynamics is modeled as a partially unknown zero-sum differential game system. Then, a periodic event-triggered optimal control algorithm is designed to interception target under a plug-n-play framework. To realize this algorithm, generalized fuzzy hyperbolic models are employed to construct the identifier-critic structure, where the online identifier is used to estimate unknown dynamics, meanwhile, the generalized fuzzy hyperbolic models-based critic network is utilized to approximate the cost function. Note that plug-n-play framework lets both the designed identifier and critic network work simultaneously, in other words, the prior system information is no longer required, which simplifies the network structure. Using the Lyapunov function method, the approximate optimal control strategy and corresponding weight updating laws are derived to guarantee that the closed-loop system and weight approximation errors are uniformly ultimately bounded, where an additional function is added to weight updating laws to release the requirement for admissible initial control. Finally, to compare the intercept effects and the utilization ratio of communication resources of the periodic event-triggered control algorithm and the common adaptive dynamic programming algorithm, the missile interception system is introduced as an example.

Machine Learning Algorithms for Detection and Classifications of Emotions in Contact Center Applications

Over the past few years, virtual assistant solutions used in Contact Center systems are gaining popularity. One of the main tasks of the virtual assistant is to recognize the intentions of the customer. It is important to note that quite often the actual intention expressed in a conversation is also directly influenced by the emotions that accompany that conversation. Unfortunately, scientific literature has not identified what specific types of emotions in Contact Center applications are relevant to the activities they perform. Therefore, the main objective of this work was to develop an Emotion Classification for Machine Detection of Affect-Tinged Conversational Contents dedicated directly to the Contact Center industry. In the conducted study, Contact Center voice and text channels were considered, taking into account the following families of emotions: anger, fear, happiness, sadness vs. affective neutrality of the statements. The obtained results confirmed the usefulness of the proposed classification-for the voice channel, the highest efficiency was obtained using the Convolutional Neural Network (accuracy, 67.5%; precision, 80.3; F1-Score, 74.5%), while for the text channel, the Support Vector Machine algorithm proved to be the most efficient (accuracy, 65.9%; precision, 58.5; F1-Score, 61.7%).

Adaptive Neural Network Stochastic-Filter-Based Controller for Attitude Tracking With Disturbance Rejection

This article proposes a real-time neural network (NN) stochastic filter-based controller on the Lie group of the special orthogonal group [Formula: see text] as a novel approach to the attitude tracking problem. The introduced solution consists of two parts: a filter and a controller. First, an adaptive NN-based stochastic filter is proposed, which estimates attitude components and dynamics using measurements supplied by onboard sensors directly. The filter design accounts for measurement uncertainties inherent to the attitude dynamics, namely, unknown bias and noise corrupting angular velocity measurements. The closed-loop signals of the proposed NN-based stochastic filter have been shown to be semiglobally uniformly ultimately bounded (SGUUB). Second, a novel control law on [Formula: see text] coupled with the proposed estimator is presented. The control law addresses unknown disturbances. In addition, the closed-loop signals of the proposed filter-based controller have been shown to be SGUUB. The proposed approach offers robust tracking performance by supplying the required control signal given data extracted from low-cost inertial measurement units. While the filter-based controller is presented in continuous form, the discrete implementation is also presented. In addition, the unit-quaternion form of the proposed approach is given. The effectiveness and robustness of the proposed filter-based controller are demonstrated using its discrete form and considering low sampling rate, high initialization error, high level of measurement uncertainties, and unknown disturbances.

An Intelligent Optimization for Building Design Based on BP Neural Network and SPEA-II Multiobjective Algorithm

With the continuous development of the field of building optimization, more and more optimization methods have sprung up, among which there are many kinds of intelligent optimization algorithms. This kind of intelligent optimization algorithm usually relies on the traditional building performance simulation method to obtain the building performance index for optimization. However, intelligent optimization algorithms generally require large-scale calculations. At the same time, the time required for building performance simulation is often limited by the complexity of building models and the configuration of computers, which leads to a long time for performance optimization, which cannot give efficient and accurate feedback to designers in engineering. Building performance optimization methods based on intelligent optimization algorithms are mainly used in scientific research and are difficult to put into practical projects. Therefore, this paper builds an accurate and efficient platform for building performance prediction and optimization to help designers make decisions combined with BP neural network and the SPEA-II multiobjective optimization algorithm. Besides, the optimization results of the case are quantitatively and qualitatively analyzed and presented in visual form based on the BP neural network prediction model. Quantitative analysis includes the evolution process of solution set, convergence process, and comprehensive quality evaluation of solution set. Qualitative analysis includes Pareto frontier and optimal architectural scheme analysis. Finally, the conclusion shows that the platform prediction and optimization can give accurate and reliable optimal solution, and the optimal building scheme is reasonable and has high engineering application value.

Trajectory Planning of Robot Manipulator Based on RBF Neural Network

Robot manipulator trajectory planning is one of the core robot technologies, and the design of controllers can improve the trajectory accuracy of manipulators. However, most of the controllers designed at this stage have not been able to effectively solve the nonlinearity and uncertainty problems of the high degree of freedom manipulators. In order to overcome these problems and improve the trajectory performance of the high degree of freedom manipulators, a manipulator trajectory planning method based on a radial basis function (RBF) neural network is proposed in this work. Firstly, a 6-DOF robot experimental platform was designed and built. Secondly, the overall manipulator trajectory planning framework was designed, which included manipulator kinematics and dynamics and a quintic polynomial interpolation algorithm. Then, an adaptive robust controller based on an RBF neural network was designed to deal with the nonlinearity and uncertainty problems, and Lyapunov theory was used to ensure the stability of the manipulator control system and the convergence of the tracking error. Finally, to test the method, a simulation and experiment were carried out. The simulation results showed that the proposed method improved the response and tracking performance to a certain extent, reduced the adjustment time and chattering, and ensured the smooth operation of the manipulator in the course of trajectory planning. The experimental results verified the effectiveness and feasibility of the method proposed in this paper.

Improved successive approximation control for formation flying at libration points of solar-earth system

In deep space exploration, the libration points (especially L2 point) of solar-earth system is a re-search hotspot in recent years. Space station and telescope can be arranged at this point, and it does not need too much kinetic energy. Therefore, it is of great significance to arrange flight formation on the libration point of solar-earth for scientific research. However, the flight keeping control technology of flight formation on the solar-earth libration points (also called Lagrange points) is one of the key problems to be solved urgently. Based on the nonlinear dynamic model of formation flying, the improved successive approximation algorithm is used to achieve formation keeping con-trol. Compared with the control algorithm based on orbital elements, this control algorithm has the advantages of high control accuracy and short control time in formation keeping control of solar-earth libration points. The disadvantage is that the calculation is complicated. But, with the devel-opment of computer technology, the computational load is gradually increasing, and there will be more extensive application value in the future. Finally, the error and control simulations of the formation flying of the spacecraft with the libration points of the solar-earth system are carried out for two days. The simulation results show that the method can quickly achieve the requirements of high-precision control.