Abstract
Reinforcement learning (RL), one of the branches of machine learning, enables a system to learn through trial and error. RL helps in solving control and decision-making tasks. Applying Deep Learning to Reinforcement learning has made it much better at solving many problems. Deep Reinforcement learning, a combination of deep learning and Reinforcement Learning is gaining a lot of interest and application in solving real-world problems. Among Deep Reinforcement learning, Deep Q networks emerged as a popular algorithm. While Deep Q networks have been successfully applied to a lot of scenarios, their application is based on the notion that the agent can completely perceive the environment. In real-time applications, this notion has a fallacy as complete observability is a difficult and sometimes impossible endeavor in real-time and the real world, therefore the use of recurrent networks along with Deep Q networks has been suggested for application to partially observable environments. This paper provides a literature review on various deep recurrent Q network applications. The paper first provides a brief introduction to the concept behind Deep Recurrent Q networks, and the various modifications to improve its performance and then proceeds to review its various applications in different fields.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. MIT Press, Cambridge (1998)
Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep Learning. MIT Press, Cambridge (2016)
Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons (2010)
Li, Y.: Deep reinforcement learning: an overview. ar**v preprint ar**v:1701.07274 (2017)
Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A.: A brief survey of deep reinforcement learning. IEEE Signal Process. Mag. 34(6), 26–38 (2017)
Lee, Y.L., Qin, D.: A survey on applications of deep reinforcement learning in resource management for 5G heterogeneous networks. In: 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 1856–1862 (2019). https://doi.org/10.1109/APSIPAASC47483.2019.9023331
Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015)
Hausknecht, M., Stone, P.: Deep recurrent q-learning for partially observable mdps. In: 2015 AAAI Fall Symposium Series (2015)
Narasimhan, K., Kulkarni, T., Barzilay, R.: Language understanding for text-based games using deep reinforcement learning. CoRR abs/1506.08941 (2015)
Zhu, P., Li, X., Poupart, P., Miao, G.: On improving deep reinforcement learning for pomdps. ar**v preprint ar**v:1704.07978 (2017)
Foerster, J.N., Assael, Y.M., de Freitas, N., Whiteson, S.: Learning to communicate to solve riddles with deep distributed recurrent q-networks. ar**v preprint ar**v:1602.02672 (2016)
Sorokin, I., Seleznev, A., Pavlov, M., Fedorov, A., Ignateva, A.: Deep attention recurrent Q-network. ar**v preprint ar**v:1512.01693 (2015)
Moreno-Vera, F.: Performing deep recurrent double Q-learning for Atari games. In: 2019 IEEE Latin American Conference on Computational Intelligence (LA-CCI), pp. 1–4 (2019). https://doi.org/10.1109/LA-CCI47412.2019.9036763
Hong, Z.-W., Su, S.-Y., Shann, T.-Y., Chang, Y.-H., Lee, C.-Y.: A deep policy inference q-network for multi-agent systems. ar**v preprint ar**v:1712.07893 (2017)
Jiajun, Ou., Guo, X., Zhu, M., Lou, W.: Autonomous quadrotor obstacle avoidance based on dueling double deep recurrent Q-learning with monocular vision. Neurocomputing 441, 300–310 (2021). https://doi.org/10.1016/j.neucom.2021.02.017
Chen, C., Ying, V., Laird, D.: Deep Q-Learning with Recurrent Neural Networks
Volodymyr, M., et al.: Playing Atari with deep reinforcement learning. In: NIPS Deep Learning Workshop (2013)
Sewak, M.: Deep Q Network (DQN), Double DQN, and Dueling DQN: a step towards general artificial intelligence. In: Sewak, M. (ed.) Deep Reinforcement Learning: Frontiers of Artificial Intelligence, pp. 95–108. Springer Singapore, Singapore (2019). https://doi.org/10.1007/978-981-13-8285-7_8
Thornton, C.E., Kozy, M.A., Buehrer, R.M., Martone, A.F., Sherbondy, K.D.: Deep reinforcement learning control for radar detection and tracking in congested spectral environments. IEEE Trans. Cognitive Commun. Netw. 6(4), 1335–1349 (2020). https://doi.org/10.1109/TCCN.2020.3019605
Xu, Y., Yu, J., Buehrer, R.M.: The application of deep reinforcement learning to distributed spectrum access in dynamic heterogeneous environments with partial observations. IEEE Trans. Wireless Commun. 19(7), 4494–4506 (2020). https://doi.org/10.1109/TWC.2020.2984227
Baek, J., Kaddoum, G.: Heterogeneous task offloading and resource allocations via deep recurrent reinforcement learning in partial observable multifog networks. IEEE Internet of Things J. 8(2), 1041–1056 (2021). https://doi.org/10.1109/JIOT.2020.3009540
Xue, S., Luo, B., Liu, D., Li, Y.: Event-triggered adaptive dynamic programming for continuous-time nonlinear two-player zero-sum game. In: Cheng, L., Leung, A.C.S., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol. 11307, pp. 15–25. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04239-4_2
Leng, S., Yener, A.: Age of information minimization for wireless ad hoc networks: a deep reinforcement learning approach. In: 2019 IEEE Global Communications Conference (GLOBECOM), pp. 1–6 (2019). https://doi.org/10.1109/GLOBECOM38437.2019.9013454
Liu, X., Zhang, H., Dong, S., Zhang, Y.: Network defense decision-making based on a stochastic game system and a deep recurrent Q-network. Comput. Secur. 111, 102480 (2021)
Chen, P., Guo, S., Gao, Y.: Deep reinforcement learning with bidirectional recurrent neural networks for dynamic spectrum access. In: 2021 IEEE 94th Vehicular Technology Conference (VTC2021-Fall), pp. 1–5 (2021). https://doi.org/10.1109/VTC2021-Fall52928.2021.9625359
Baek, J., Kaddoum, G.: Online partial offloading and task scheduling in SDN-fog networks with deep recurrent reinforcement learning. IEEE Internet of Things J. 9, 11578–11589 (2021). https://doi.org/10.1109/JIOT.2021.3130474
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Gayatri Shivani, M.V.K., Subba Rao, S.P.V., Sujatha, C.N. (2023). A Survey on Deep Recurrent Q Networks. In: Nandan Mohanty, S., Garcia Diaz, V., Satish Kumar, G.A.E. (eds) Intelligent Systems and Machine Learning. ICISML 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 470. Springer, Cham. https://doi.org/10.1007/978-3-031-35078-8_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-35078-8_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35077-1
Online ISBN: 978-3-031-35078-8
eBook Packages: Computer ScienceComputer Science (R0)