Reinforcement Learning-Based Anti-disturbances Adaptive Control for Systems Subjected to Mismatched Disturbances and Input Uncertainties

Abstract

This paper studies the anti-disturbances adaptive control problem with reinforcement learning (RL) actor-critic method for systems which subjected to matched, mismatched disturbances and input uncertainties. As most of the classical adaptive methods are not applicable in this case, firstly, actor-critic networks are introduced to approximate the unknown dynamics and cost function respectively. And the critic network is used to judge the performance of the actor network and give reinforcement signal to guide the updating of network weights. Furthermore, by using the hyperbolic tangent function to estimate the disturbances boundaries, the input uncertainties and time-varying disturbances can be matched and solved. As a result, an adaptive controller and a series of adaptive parameter update laws based on the backstep** method are proposed, which can accelerate the convergence under multi-source uncertainties without priori information. It also overcomes the shortcoming of data-based reinforcement learning not guaranteeing stability. Finally, through analyzing the Lyapunov function, the controller is proved to be actual exponential stable and all kinds of errors are bounded. The numerical simulation shows the validity and superiority of the proposed method.