Abstract
Autonomous robots are not very good at being autonomous. They work well in structured environments, but fail quickly in the real world facing uncertainty and dynamically changing conditions. In this chapter, we describe robot learning approaches that help to elevate robot autonomy to the next level, the so-called ‘persistent autonomy’. For a robot to be ‘persistently autonomous’ means to be able to perform missions over extended time periods (e.g. days or months) in dynamic, uncertain environments without need for human assistance. In particular, persistent autonomy is extremely important for robots in difficult-to-reach environments such as underwater, rescue, and space robotics. There are many facets of persistent autonomy, such as: co** with uncertainty, reacting to changing conditions, disturbance rejection, fault tolerance, energy efficiency and so on. This chapter presents a collection of robot learning approaches that address many of these facets. Experiments with robot manipulators and autonomous underwater vehicles demonstrate the usefulness of these learning approaches in real world scenarios.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abidi MA, Eason RO, Gonzalez RC (1991) Autonomous robotic inspection and manipulation using multisensor feedback. Computer 24(4):17–31
Ahmadzadeh SR, Kormushev P, Caldwell DG (2013a) Autonomous robotic valve turning: a hierarchical learning approach. In: 2013 IEEE international conference on robotics and automation (ICRA). IEEE, pp 4614–4619
Ahmadzadeh SR, Leonetti M, Kormushev P (2013b) Online direct policy search for thruster failure recovery in autonomous underwater vehicles. In: 6th international workshop on evolutionary and reinforcement learning for autonomous robot system (ERLARS 2013), Taormina, Italy
Ahmadzadeh SR, Jamisola RS, Kormushev P, Caldwell DG (2014a) Learning reactive robot behavior for autonomous valve turning. In: Proceedings of the IEEE international conference on humanoid robots (Humanoids 2014), Madrid, Spain
Ahmadzadeh SR, Leonetti M, Carrera A, Carreras M, Kormushev P, Caldwell DG (2014b) Online discovery of AUV control policies to overcome thruster failure. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE, pp 6522–6528
Ajoudani A, Lee J, Rocchi A, Ferrati M, Mingo E, Settimi A, Caldwell DG, Bicchi A, Tsagarakis N (2014) A manipulation framework for compliant humanoid COMAN: application to a valve turning task. In: 2014 IEEE-RAS international conference on humanoid robots (Humanoids 2014). IEEE, pp 664–670
Alessandri A, Caccia M, Veruggio G (1998) A model-based approach to fault diagnosis in unmanned underwater vehicles. In: OCEANS’98 conference proceedings, vol 2. IEEE, pp 825–829
Alunni N, Phillips-Grafftin C, Suay HB, Lofaro D, Berenson D, Chernova S, Lindeman RW, Oh P (2013) Toward a user-guided manipulation framework for high-dof robots with limited communication. In: 2013 IEEE international conference on technologies for practical robot applications (TePRA). IEEE, pp 1–6
Anisi DA, Persson E, Heyer C (2011) Real-world demonstration of sensor-based robotic automation in oil & gas facilities. In: 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 235–240
Anisi DA, Skourup C, Petrochemicals A (2012) A step-wise approach to oil and gas robotics. In: IFAC workshop on automatic control in offshore oil and gas production, Trondheim, Norway, vol 31
Antonelli G (2003) A survey of fault detection/tolerance strategies for AUVs and ROVs. In: Caccavale F, Villani L (eds) Fault diagnosis and fault tolerance for mechatronic systems: recent advances. Springer, Berlin, pp 109–127
Antonelli G (2006) Underwater robots: motion and force control of vehicle-manipulator systems. Springer tracts in advanced robotics. Springer, New York
Bristow D, Tharayil M, Alleyne AG et al (2006) A survey of iterative learning control. IEEE Control Syst 26(3):96–114
Caccia M, Bono R, Bruzzone G, Bruzzone G, Spirandelli E, Veruggio G (2001) Experiences on actuator fault detection, diagnosis and accomodation for ROVs. In: International symposiyum of unmanned untethered sub-mersible technology
Carrera A, Ahmadzadeh S, Ajoudani A, Kormushev P, Carreras M, Caldwell D (2012) Towards autonomous robotic valve turning. Cybern Inf Technol 12(3):17–26
Cheng ASF, Leonard NE (1999) Fin failure compensation for an unmanned underwater vehicle. In: Proceedings of the 11th international symposium on unmanned untethered submersible technology
Das SN, Das SK (2004) Determination of coupled sway, roll, and yaw motions of a floating body in regular waves. Int J Math Math Sci 41:2181–2197
Hamilton K, Lane D, Taylor N, Brown K (2001) Fault diagnosis on autonomous robotic vehicles with recovery: an integrated heterogeneous-knowledge approach. In: Proceedings of the IEEE International Conference on Robotics and Automation, ICRA, 2001, vol 4. IEEE, pp 3232–3237
Ijspeert AJ, Nakanishi J, Hoffmann H, Pastor P, Schaal S (2013) Dynamical movement primitives: learning attractor models for motor behaviors. Neural Comput 25(2):328–373
Jamisola RS, Oetomo DN, Ang MH, Khatib O, Lim TM, Lim SY (2005) Compliant motion using a mobile manipulator: an operational space formulation approach to aircraft canopy polishing. Adv Robot 19(5):613–634
Karras GC, Bechlioulis CP, Leonetti M, Palomeras N, Kormushev P, Kyriakopoulos KJ, Caldwell DG (2013) On-line identification of autonomous underwater vehicles through global derivative-free optimization. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS)
Khatib O (1987) A unified approach for motion and force control of robot manipulators: the operational space formulation. IEEE J Robot Autom 3(1):43–53
Kirkpatrick S, Gelatt CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680
Konidaris G, Osentoski S, Thomas PS (2011) Value function approximation in reinforcement learning using the fourier basis. In: AAAI
Kormushev P, Caldwell DG (2013a) Improving the energy efficiency of autonomous underwater vehicles by learning to model disturbances. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS), Tokyo, Japan
Kormushev P, Caldwell DG (2013b) Towards improved AUV control through learning of periodic signals. In: Proceedings of the MTS/IEEE international conference on OCEANS 2013, San Diego
Kormushev P, Calinon S, Caldwell DG (2011) Imitation learning of positional and force skills demonstrated via kinesthetic teaching and haptic input. Adv Robot 25(5):581–603
Lane DM, Maurelli F, Kormushev P, Carreras M, Fox M, Kyriakopoulos K (2012) Persistent autonomy: the challenges of the PANDORA project. In: Proceedings of the IFAC MCMC
Leonetti M, Kormushev P, Sagratella S (2012) Combining local and global direct derivative-free optimization for reinforcement learning. Cybern Inf Technol 12(3):53–65
Leonetti M, Ahmadzadeh SR, Kormushev P (2013) On-line learning to recover from thruster failures on autonomous underwater vehicles. In: OCEANS 2013. IEEE
Moore KL (2012) Iterative learning control for deterministic systems. Springer Science & Business Media, London
Orsag M, Korpela C, Bogdan S, Oh P (2014) Valve turning using a dual-arm aerial manipulator. In: 2014 international conference on unmanned aircraft systems (ICUAS). IEEE, pp 836–841
PANDORA (2012) Persistent autonomy through learning, adaptation, observation and re-planning. http://persistentautonomy.com/, PANDORA European Project
Perrault D, Nahon M (1998) Fault-tolerant control of an autonomous underwater vehicle. In: OCEANS’98 conference proceedings, vol 2. IEEE, pp 820–824
Podder T, Antonelli G, Sarkar N (2000) Fault tolerant control of an autonomous underwater vehicle under thruster redundancy: simulations and experiments. In: Proceedings of the IEEE international conference on robotics and automation, ICRA’00, 2000, vol 2. IEEE, pp 1251–1256
Podder TK, Sarkar N (2001) Fault-tolerant control of an autonomous underwater vehicle under thruster redundancy. Robot Auton Syst 34(1):39–52
Raibert MH, Craig JJ (1981) Hybrid position/force control of manipulators. J Dyn Syst, Measur, Control 103(2):126–133
Ribas D, Palomeras N, Ridao P, Carreras M, Mallios A (2012) Girona 500 AUV: from survey to intervention. IEEE/ASME Trans Mechatron 17(1):46–53
Schaal S, Ijspeert A, Billard A (2003) Computational approaches to motor learning by imitation. Philoso Trans R Soc Lond Ser B: Biol Sci 358(1431):537–547
Seto ML (2011) An agent to optimally re-distribute control in an underactuated AUV. Int J Intell Def Support Syst 4(1):3–19
Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359
Wang L (1999) A course on fuzzy systems. Prentice-Hall press, Upper Saddle River
Yoshikawa T, Zheng XZ (1993) Coordinated dynamic hybrid position/force control for multiple robot manipulators handling one constrained object. Int J Robot Res 12(3):219–230
Acknowledgments
We would like to thank Professor David Lane from the Ocean Systems Laboratory, Heriot-Watt University, UK, for introducing us to the topic of persistent autonomy.
We are grateful to Arnau Carrera, Narcís Palomeras, and Marc Carreras from the Computer Vision and Robotics Group (VICOROB), University of Girona, Spain, for making it possible to conduct real-world experiments with the Girona 500 AUV.
This work was supported by the European project PANDORA: Persistent Autonomy through learNing, aDaptation, Observation and ReplAnning, contract FP7-ICT-288273 (PANDORA 2012).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Kormushev, P., Ahmadzadeh, S.R. (2015). Robot Learning for Persistent Autonomy. In: Busoniu, L., Tamás, L. (eds) Handling Uncertainty and Networked Structure in Robot Control. Studies in Systems, Decision and Control, vol 42. Springer, Cham. https://doi.org/10.1007/978-3-319-26327-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-26327-4_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26325-0
Online ISBN: 978-3-319-26327-4
eBook Packages: EngineeringEngineering (R0)