Abstract
In the field of robotics, it is a challenge to deal with multi-stage tasks based on Deep reinforcement learning (Deep RL). Previous researches have shown manually sha** a reward function could easily result in sub-optimal performance, hence choosing a sparse reward is a natural and sensible decision in many cases. However, it is rare for the agent to explore a non-zero reward with the increase of the horizon under the sparse reward, which makes it difficult to learn an agent to deal with multi-stage task. In this paper, we aim to develop a Deep RL based policy through fully utilizing the demonstrations to address this problem. We use the learned policy to solve some difficult multi-stage tasks, such as picking-and-place, stacking blocks, and achieve good results. A video of our experiments can be found at: https://youtu.be/6BulNjqDg3I.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Andrychowicz, M., et al.: Hindsight experience replay. In: Advances in Neural Information Processing Systems, pp. 5048–5058 (2017)
Coumans, E., Bai, Y.: Pybullet, a python module for physics simulation for games, robotics and machine learning. GitHub repository (2016)
Hester, T., et al.: Learning from demonstrations for real world reinforcement learning. ar**v preprint ar**v:1704.03732 (2017)
Levine, S., Finn, C., Darrell, T., Abbeel, P.: End-to-end training of deep visuomotor policies. J. Mach. Learn. Res. 17(1), 1334–1373 (2016)
Matas, J., James, S., Davison, A.J.: Sim-to-real reinforcement learning for deformable object manipulation. ar**v preprint ar**v:1806.07851 (2018)
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., Abbeel, P.: Overcoming exploration in reinforcement learning with demonstrations. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6292–6299. IEEE (2018)
Pinto, L., Andrychowicz, M., Welinder, P., Zaremba, W., Abbeel, P.: Asymmetric actor critic for image-based robot learning. ar**v preprint ar**v:1710.06542 (2017)
Pinto, L., Gupta, A.: Supersizing self-supervision: learning to grasp from 50k tries and 700 robot hours. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3406–3413. IEEE (2016)
Popov, I., et al.: Data-efficient deep reinforcement learning for dexterous manipulation. ar** using simulated depth images. ar**v preprint ar**v:1706.04652 (2017)
Acknowledgements
This work was supported in part by NSFC under Grant No.91848109, supported by Bei**g Natural Science Foundation under Grant No.4182068 and supported by Science and Technology on Space Intelligent Control Laboratory under No. HTKJ2019KL502013.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, B., Su, J. (2019). Addressing Reward Engineering for Deep Reinforcement Learning on Multi-stage Task. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Communications in Computer and Information Science, vol 1143. Springer, Cham. https://doi.org/10.1007/978-3-030-36802-9_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-36802-9_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36801-2
Online ISBN: 978-3-030-36802-9
eBook Packages: Computer ScienceComputer Science (R0)