Abstract
This work extends an existing virtual multi-agent platform called RoboSumo to create TripleSumo—a platform for investigating multi-agent cooperative behaviors in continuous action spaces, with physical contact in an adversarial environment. In this paper we investigate a scenario in which two agents, namely ‘Bug’ and ‘Ant’, must team up and push another agent ‘Spider’ out of the arena. To tackle this goal, the newly added agent ‘Bug’ is trained during an ongoing match between ‘Ant’ and ‘Spider’. ‘Bug’ must develop awareness of the other agents’ actions, infer the strategy of both sides, and eventually learn an action policy to cooperate. The reinforcement learning algorithm Deep Deterministic Policy Gradient (DDPG) is implemented with a hybrid reward structure combining dense and sparse rewards. The cooperative behavior is quantitatively evaluated by the mean probability of winning the match and mean number of steps needed to win.
This work benefited from the advice of Assistant Professor Shinkyu Park at King Abdullah University of Science and Technology (KAUST).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
supplementary video 1: https://www.youtube.com/watch?v=nxzi7Pha2GU.
- 4.
supplementary video 2: https://www.youtube.com/watch?v=C4xGuIyeY5A.
References
Yin, Q., Yang, J., Ni, W., Liang, B., Huang, K.: AI in games: techniques, challenges and opportunities. Ar**v, vol. abs/2111.07631 (2021)
Liu, S., et al.: From motor control to team play in simulated humanoid football. Ar**v, vol. abs/2105.12196 (2021)
Al-Shedivat, M., Bansal, T., Burda, Y., Sutskever, I., Mordatch, I., Abbeel, P.: Continuous adaptation via meta-learning in nonstationary and competitive environments. Ar**v, vol. abs/1710.03641 (2018)
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning. CoRR, vol. abs/1509.02971 (2016)
Kurach, K., et al.: Google research football: a novel reinforcement learning environment. In: AAAI (2020)
Huang, S., et al.: TiKick: towards playing multi-agent football full games from single-agent demonstrations. Ar**v, vol. abs/2110.04507 (2021)
Jia, H., et al.: Fever basketball: a complex, flexible, and asynchronized sports game environment for multi-agent reinforcement learning. Ar**v, vol. abs/2012.03204 (2020)
Baker, B., et al.: Emergent tool use from multi-agent autocurricula. Ar**v, vol. abs/1909.07528 (2020)
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments. Ar**v, vol. abs/1706.02275 (2017)
Zhang, T., et al.: Multi-agent collaboration via reward attribution decomposition. Ar**v, vol. abs/2010.08531 (2020)
de Souza, C., Newbury, R., Cosgun, A., Castillo, P., Vidolov, B., Kulić, D.: Decentralized multi-agent pursuit using deep reinforcement learning. IEEE Rob. Autom. Lett. 6, 4552–4559 (2021)
Von Moll, A., Casbeer, D.W., Garcia, E., Milutinović, D.: Pursuit-evasion of an evader by multiple pursuers. In: 2018 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 133–142 (2018)
Brockman, G., et al.: OpenAI gym. Ar**v, vol. abs/1606.01540 (2016)
Todorov, E., Erez, T., Tassa, Y.: MuJoCo: a physics engine for model-based control. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5026–5033 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, N., Das, G.P., Millard, A.G. (2022). Learning Cooperative Behaviours in Adversarial Multi-agent Systems. In: Pacheco-Gutierrez, S., Cryer, A., Caliskanelli, I., Tugal, H., Skilton, R. (eds) Towards Autonomous Robotic Systems. TAROS 2022. Lecture Notes in Computer Science(), vol 13546. Springer, Cham. https://doi.org/10.1007/978-3-031-15908-4_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-15908-4_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15907-7
Online ISBN: 978-3-031-15908-4
eBook Packages: Computer ScienceComputer Science (R0)