Abstract
The RoboCup soccer simulation 2D domain is a very large testbed for the research of planning and machine learning. It has competed in the annual world championship tournaments in the past 15 years. However it is still unclear that whether more principled techniques such as decision-theoretic planning take an important role in the success for a RoboCup 2D team. In this paper, we present a novel approach based on MAXQ-OP to automated planning in the RoboCup 2D domain. It combines the benefits of a general hierarchical structure based on MAXQ value function decomposition with the power of heuristic and approximate techniques. The proposed framework provides a principled solution to programming autonomous agents in large stochastic domains. The MAXQ-OP framework has been implemented in our RoboCup 2D team, WrightEagle. The empirical results indicated that the agents developed with this framework and related techniques reached outstanding performances, showing its potential of scalability to very large domains.
Chapter PDF
Similar content being viewed by others
References
Bai, A., Wu, F., Chen, X.: Online planning for large MDPs with MAXQ decomposition (extended abstract). In: Proc. of 11th Int. Conf. on Autonomous Agents and Multiagent Systems, Valencia, Spain (June 2012)
Barry, J.: Fast Approximate Hierarchical Solution of MDPs. Ph.D. thesis, Massachusetts Institute of Technology (2009)
Barry, J., Kaelbling, L., Lozano-Perez, T.: Deth*: Approximate hierarchical solution of large markov decision processes. In: International Joint Conference on Artificial Intelligence, pp. 1928–1935 (2011)
Dietterich, T.G.: Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of Machine Learning Research 13(1), 63 (May 1999)
Gabel, T., Riedmiller, M.: On progress in roboCup: The simulation league showcase. In: Ruiz-del-Solar, J. (ed.) RoboCup 2010. LNCS (LNAI), vol. 6556, pp. 36–47. Springer, Heidelberg (2011)
Kalyanakrishnan, S., Liu, Y., Stone, P.: Half field offense in roboCup soccer: A multiagent reinforcement learning case study. In: Lakemeyer, G., Sklar, E., Sorrenti, D.G., Takahashi, T. (eds.) RoboCup 2006. LNCS (LNAI), vol. 4434, pp. 72–85. Springer, Heidelberg (2007)
Riedmiller, M., Gabel, T., Hafner, R., Lange, S.: Reinforcement learning for robot soccer. Autonomous Robots 27(1), 55–73 (2009)
Stone, P.: Layered learning in multiagent systems: A winning approach to robotic soccer. The MIT press (2000)
Stone, P., Sutton, R., Kuhlmann, G.: Reinforcement learning for robocup soccer keepaway. Adaptive Behavior 13(3), 165–188 (2005)
Thrun, S., Fox, D., Burgard, W., Dellaert, F.: Robust monte carlo localization for mobile robots. Artificial Intelligence 128(1-2), 99–141 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bai, A., Wu, F., Chen, X. (2013). Towards a Principled Solution to Simulated Robot Soccer. In: Chen, X., Stone, P., Sucar, L.E., van der Zant, T. (eds) RoboCup 2012: Robot Soccer World Cup XVI. RoboCup 2012. Lecture Notes in Computer Science(), vol 7500. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39250-4_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-39250-4_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39249-8
Online ISBN: 978-3-642-39250-4
eBook Packages: Computer ScienceComputer Science (R0)