Abstract
A wireless sensor network (WSN) consists of a large number of sensor nodes with limited battery lives that are dispersed geographically to monitor events and gather information from a geographical area. On the other hand, tactical WSNs are mission-critical WSNs that are used to support military operations, such as intrusion detection, battlefield surveillance, and combat monitoring. Such networks are critical to the collection of situational data on a battlefield for timely decision-making. Due to their application area, tactical WSNs have unique challenges, not seen in commercial WSNs, such as being targets for adversarial attacks. These challenges make packet routing in tactical WSNs a daunting task. In this article, we propose a multi-agent Q-learning-based routing scheme for a tactical WSN consisting of static sensors and a mobile sink. Using the proposed routing scheme, a learning agent (i.e., network node) adjusts its routing policy according to the estimates of the Q-values of the available routes via its neighbors. The Q-values capture the quickness, reliability, and energy efficiency of the routes as a function of the number of hops to sink, the one-hop delay, the energy cost of transmission, and the packet loss rate of the neighbors. Simulation results demonstrate that, in comparison to a baseline random hop selection scheme, the proposed scheme reduces the packet loss rate and mean hop delay, and enhances energy efficiency in the presence of jamming attacks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Thulasiraman, P., White, K.A.: Topology control of tactical wireless sensor networks using energy efficient zone routing. Digit. Commun. Netw. 2(1), 1–14 (2016)
Mutombo, V.K., Lee, S., Lee, J., Hong, J.: EER-RL: energy-efficient routing based on reinforcement learning. Mob. Inf. Syst. 2021, 1–12 (2021)
Ghosh, N., Banerjee, I.: Application of mobile sink in wireless sensor networks. In: 2018 10th International Conference on Communication Systems & Networks (COMSNETS), pp. 507–509. IEEE, Bengaluru, India (2018)
Jain, S., Verma, R.K., Pattanaik, K.K., Shukla, A.: A survey on event-driven and query-driven hierarchical routing protocols for mobile sink-based wireless sensor networks. J. Supercomput. 78(9), 11492–11538 (2022)
Krishnan, M., Lim, Y.: Reinforcement learning-based dynamic routing using mobile sink for data collection in WSNs and IoT applications. J. Netw. Comput. Appl. 194, 103223 (2021)
Keum, D., Ko, Y.B.: Trust-based intelligent routing protocol with q-learning for mission-critical wireless sensor networks. Sensors 22(11), 3975 (2022)
Naghibi, M., Barati, H.: EGRPM: energy efficient geographic routing protocol based on mobile sink in wireless sensor networks. Sustain. Comput. Inform. Syst. 25, 100377 (2020)
Wang, J., Gao, Y., Liu, W., Sangaiah, A.K., Kim, H.J.: Energy efficient routing algorithm with mobile sink support for wireless sensor networks. Sensors 19(7), 1494 (2019)
Li, X., Yang, J., Nayak, A., Stojmenovic, I.: Localized geographic routing to a mobile sink with guaranteed delivery in sensor networks. IEEE J. Sel. Areas Commun. 30(9), 1719–1729 (2012)
Jain, S., Pattanaik, K.K., Verma, R.K., Bharti, S., Shukla, A.: Delay-aware green routing for mobile-sink-based wireless sensor networks. IEEE Internet Things J. 8(6), 4882–4892 (2020)
Tunca, C., Isik, S., Donmez, M.Y., Ersoy, C.: Ring routing: an energy-efficient routing protocol for wireless sensor networks with a mobile sink. IEEE Trans. Mob. Comput. 14(9), 1947–1960 (2014)
Mitra, R., Sharma, S.: Proactive data routing using controlled mobility of a mobile sink in wireless sensor networks. Comput. Electri. Eng. 70, 21–36 (2018)
Guo, W., Yan, C., Lu, T.: Optimizing the lifetime of wireless sensor networks via reinforcement-learning-based routing. Int. J. Distrib. Sens. Netw. 15(2), 1550147719833541 (2019)
Yun, W.K., Yoo, S.J.: Q-learning-based data-aggregation-aware energy-efficient routing protocol for wireless sensor networks. IEEE Access 9, 10737–10750 (2021)
Bouzid, S. E., Serrestou, Y., Raoof, K., Omri, M. N.: Efficient routing protocol for wireless sensor network based on reinforcement learning. In: 2020 5th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp. 1–5. IEEE, Sousse, Tunisia (2020)
Abadi, A.F.E., Asghari, S.A., Marvasti, M.B., Abaei, G., Nabavi, M., Savaria, Y.: RLBEEP: Reinforcement-Learning-Based Energy Efficient Control and Routing Protocol for Wireless Sensor Networks. IEEE Access 10, 44123–44135 (2022)
Obi, E., Mammeri, Z., Ochia, O. E.: A Lifetime-Aware Centralized Routing Protocol for Wireless Sensor Networks using Reinforcement Learning. In: 2021 17th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), pp. 363–368. IEEE, Bologna, Italy (2021)
Haakensen, T., Thulasiraman, P.: Enhancing sink node anonymity in tactical sensor networks using a reactive routing protocol. In: 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON), pp. 115–121. IEEE, New York (2017)
Nguyen, S. T., Cayirci, E., Yan, L., Rong, C.: A shadow zone aware routing protocol for tactical acoustic undersea surveillance networks. In: MILCOM 2009–2009 IEEE Military Communications Conference, pp. 1–7. IEEE, Boston (2009)
Altowaijri, S.M.: Efficient next-hop selection in multi-hop routing for iot enabled wireless sensor networks. Fut. Internet 14(2), 35 (2022)
Liu, Lei, Liu, Yiming, Wang, Zhaowei, Liu, Chunxu: Design of dynamic tdma protocols for tactical data link. In: Li, Bo., Shu, Lei, Zeng, Deze (eds.) ChinaCom 2017. LNICST, vol. 236, pp. 166–175. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-78130-3_18
Sheikholeslami, A., Pishro-Nik, H., Ghaderi, M., Goeckel, D.: On the impact of dynamic jamming on end-to-end delay in linear wireless networks. In: 2014 48th Annual Conference on Information Sciences and Systems (CISS), pp. 1–6. IEEE, Princeton, NJ (2014)
Majumdar, S., Trivisonno, R., Carle, G.: Understanding Exploration and Exploitation of Q-Learning Agents in B5G Network Management. In: 2021 IEEE Globecom Workshops (GC Wkshps), pp. 1–6. IEEE, Madrid (2021)
Gao, D., Liu, Y., Zhang, F., Song, J.: Anycast routing protocol for forest monitoring in rechargeable wireless sensor networks. Int. J. Distrib. Sens. Netw. 9(12), 239860 (2013)
Wilson, Graeme N.., Ramirez-Serrano, Alejandro, Mustafa, Mahmoud, Davies, Krispin A..: Velocity selection for high-speed ugvs in rough unknown terrains using force prediction. In: Su, Chun-Yi., Rakheja, Subhash, Liu, Honghai (eds.) ICIRA 2012. LNCS (LNAI), vol. 7507, pp. 387–396. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33515-0_39
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Okine, A.A., Adam, N., Kaddoum, G. (2023). Reinforcement Learning Aided Routing in Tactical Wireless Sensor Networks. In: Sabir, E., Elbiaze, H., Falcone, F., Ajib, W., Sadik, M. (eds) Ubiquitous Networking. UNet 2022. Lecture Notes in Computer Science, vol 13853. Springer, Cham. https://doi.org/10.1007/978-3-031-29419-8_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-29419-8_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-29418-1
Online ISBN: 978-3-031-29419-8
eBook Packages: Computer ScienceComputer Science (R0)