Abstract
A Q-learning scheme is applied to the control of the aircraft system, which solves the infinite-horizon linear quadratic regulator (LQR) problem and linear quadratic tracking (LQT) problem without requiring the knowledge of system dynamics. For regulation problem, the solution depends on the Q-function based Bellman equation. There is no need for system dynamics to solve Bellman equation by using value iteration (VI) algorithm. For tracking problem, an augmented system composed of the original system and the reference trajectory system is built. The related Q-function based Bellman equation is established. By employing Q-learning, the tracking problem is computed online without requiring the knowledge of the augmented system dynamics. Finally, the effectiveness of the proposed methods is verified by the simulations on the longitudinal model of the F-16 aircraft.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Babaei, A.R., Mortazavi, M., Moradi, M.H.: Classical and fuzzy-genetic autopilot design for unmanned aerial vehicles. Appl. Soft Comput. 11(1), 365–372 (2011)
Huang, S., Huang, J., Cai, Z., et al.: Adaptive backstep** sliding mode control for quadrotor UAV. Scientific Programming (2021)
He, J.L., Jiang, R.B., Liu, B., et al.: The study and design of UAV dynamic inversion flight control law. Adv. Mat. Res. 139–141, 1757–1762 (2010)
Fang, Z., Hao, C., Li, P.: Model reference output feedback control using episodic natural actor-critic. In: 2012 IEEE International Symposium on Industrial Electronics, pp. 1286–1290. Hangzhou (2012)
Bouammar, H., Voos, H., Ertel, W.: Controller design for quadrotor UAVs using reinforcement learning. In: IEEE International Conference on Control Applications. pp. 2130–2135. Yokohama (2010)
Zhou, Y., Kampen, E., Chu, Q.P.: Nonlinear adaptive flight control using incremental approximate dynamic programming and output feedback. J. Guid. Control Dyn. 40(2), 1–8 (2016)
Nguyen, L.T.: Simulator study of stall/post-stall characteristics of a fighter airplane with relaxed longitudinal static stability. National Aeronautics and Space Administration (1979)
Stevens, B.L., Lewis, F.L., Johnson, E.N.: Aircraft Control and Simulation: Dynamics, Controls Design, and Autonomous Systems. Wiley (2015)
Acknowledgements
This work has been supported in part by the National Natural Science Foundation of China under Grant 62003161, in part by Natural Science Foundation of Jiangsu Province under Grant BK20190399, in part by the China Postdoctoral Science Foundation under Grant 2021M701701, in part by the Fundamental Research Funds for the Central Universities under Grant NS2020020, and in part by the Research Funds for the Engineering Research Center of Aircraft Autonomous Control Technology, Ministry of Education, under Grant NJ2020004.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Cheng, L., Zhao, Z., Kong, F. (2023). Q-Learning Based Online Adaptive Flight Control for Fixed-Wing UAV. In: Yan, L., Duan, H., Deng, Y. (eds) Advances in Guidance, Navigation and Control. ICGNC 2022. Lecture Notes in Electrical Engineering, vol 845. Springer, Singapore. https://doi.org/10.1007/978-981-19-6613-2_7
Download citation
DOI: https://doi.org/10.1007/978-981-19-6613-2_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-6612-5
Online ISBN: 978-981-19-6613-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)