Abstract
This paper aims to introduce nonlinear optimization in the fuzzy reinforcement learning (RL) approach through genetic algorithm (GA)-based minimization. In conventional fuzzy RL, an agent attempts to find most optimal action at each stage by choosing an action having the lowest Q value or the greedy action. However, Q function is an unknown function and an attempt to find minima of such a function based on a limited set of values, in our view, is inaccurate and insufficient. A more rigorous approach would be to employ a nonlinear optimization procedure for finding minima of the Q function. We propose to employ genetic algorithm for finding optimal action value in each iteration of the algorithm rather than plain algebraic minimum. For guaranteed stability of the designed controller, we use Lyapunov theory-based fuzzy RL control with GA optimizer. We validate the performance of our controller on three benchmark nonlinear NL control problems: (1) inverted pendulum swing up, (2) cart pole balance, and (3) rotational/translational proof-mass actuator system. We carry out comparative evaluation of our controller against: (1) hybrid Lyapunov fuzzy RL control and (2) fuzzy Q learning control. Results show that our proposed GA-optimized fuzzy Lyapunov RL controller is able to achieve a high success rate with stable and superior tracking performance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Russell, S.J.; Norvig, P.: Artificial Intelligence: A Modern Approach. Pearson Education Limited, Kuala Lumpur (2016)
Sammut, C.; Webb, G.I.: Encyclopedia of Machine Learning and Data Mining. Springer, Berlin (2017)
Mars, P.: Learning Algorithms: Theory and Applications in Signal Processing, Control and Communications. CRC Press, Boca Raton (2018)
Silver, D.; et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484 (2016)
Wang, H.; et al.: Adaptive neural tracking control for a class of nonlinear systems with dynamic uncertainties. IEEE Trans. Cybern. 47(10), 3075–3087 (2017)
Baghya Shree, S.; Kamaraj, N.: Hybrid neuro fuzzy approach for automatic generation control in restructured power system. Int. J. Electr. Power Energy Syst. 74, 274–285 (2016)
Mannion, P. et al.: Dynamic economic emissions dispatch optimisation using multi-agent reinforcement learning. In: Proceedings of the Adaptive and Learning Agents Workshop (at AAMAS 2016). (2016)
Liu, Y.; et al.: Partial-nodes-based state estimation for complex Networks with unbounded distributed delays. IEEE Trans. Neural Netw. Learn. Syst. 29(8), 3906–3912 (2018)
Rubio, J.J.: Modified optimal control with a back propagation network for robotic arms. IET Control Theory Appl. 6(14), 2216–2225 (2012)
de Jes´us Rubio, J.: SOFMLS: online self-organizing fuzzy modified least-squares network. IEEE Trans. Fuzzy Syst. 17(6), 1296–1309 (2009)
Li, X.; et al.: Assessing information security risk for an evolving smart city based on fuzzy and grey FMEA. J Intell Fuzzy Syst. 34(4), 2491–2501 (2018)
de Jes´us Rubio, J.; et al.: Neural network updating via argument Kalman filter for modeling of Takagi–Sugeno fuzzy models. J. Intell. Fuzzy Syst. 35(2), 2585–2596 (2018)
Yen, V.T.; Nan, W.Y.; Van Cuong, P.: Recurrent fuzzy wavelet neural networks based on robust adaptive sliding mode control for industrial robot manipulators. Neural Comput. Appl. (2018). https://doi.org/10.1007/s00521-018-3520-3
Yang, Z.; Ce, L.; Lian, L.: Electricity price forecasting by a hybrid model, combining wavelet transform, ARMA and kernel-based extreme learning machine methods. Appl. Energy 190, 291–305 (2017)
Mohri, M.; Rostamizadeh, A.; Talwalkar, A.: Foundations of Machine Learning. MIT Press, Cambridge (2012)
Sharma, R.: Lyapunov theory based stable Markov game fuzzy control for non-linear systems. Eng. Appl. Artif. Intell. 55, 119–127 (2016)
Kumar, A.; Sharma, R.: Fuzzy lyapunov reinforcement learning for non linear systems. ISA Trans. 67, 151–159 (2017)
Aguilar-Ibanez, C.: A constructive Lyapunov function for controlling the inverted pendulum. In: American Control Conference, Westin Seattle Hotel, Seattle, Washington, USA, 11–13 June 2008
Zhang, Q.; et al.: Energy-efficient scheduling for real-time systems based on deep Q-learning model. IEEE Trans. Sustain. Comput. 4, 132–141 (2017)
Deng, Y.; et al.: Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 653–664 (2017)
Such, F.P., et al.: Deep neuroevolution: genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning (2017). ar**v preprint ar**v:1712.06567
Kumar, A.; Sharma, R.: Neural/fuzzy self learning Lyapunov control for non linear systems. Int. J. Inf. Technol. (2018). https://doi.org/10.1007/s41870-017-0074-z
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kukker, A., Sharma, R. Genetic Algorithm-Optimized Fuzzy Lyapunov Reinforcement Learning for Nonlinear Systems. Arab J Sci Eng 45, 1629–1638 (2020). https://doi.org/10.1007/s13369-019-04126-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13369-019-04126-9