Abstract
This study proposes a path planning algorithm for marine vehicles based on machine learning. The algorithm considers the dynamic characteristics of the vehicle and disturbance effects in ocean environments. The movements of marine vehicles are influenced by various physical disturbances in ocean environments, such as wind, waves, and currents. In the present study, the effects of ocean currents are the primary consideration. A kinematic model is used to incorporate the nonholonomic motion characteristics of a marine vehicle, and the reinforcement learning algorithm is used for path optimization to generate a feasible path that can be tracked by the vehicle. The proposed approach determines a near-optimal path that connects the start and goal points with a reasonable computational cost when the map and current field data are provided. To verify the optimality and validity of the proposed algorithm, a set of simulations were performed in simulated and actual ocean current conditions, and their results are presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bryson AE, Ho YC (1975) Applied Optimal Control: Optimization. Estimation and Control. Hemisphere Pub, Corp
Sutton RS, Barto AG (1998) Introduction to Reinforcement Learning. MIT Press
Warren CW (1990) A technique for autonomous underwater vehicle route planning. IEEE J Ocean Eng 15(3):199–204
Carroll KP, McClaran SR, Nelson EL, Barnett DM, Friesen DK, Williams GN (1992) AUV path planning: an A* approach to path planning with consideration of variable vehicle speeds and multiple, overlap**, time-dependent exclusion zones. In: Proceedings of the Symposium on AUV Technology, Washington, DC, June 02–03, pp 79–84
Sugihara K, Yuh J (1997) GA-based motion planning for underwater robotic vehicle. In: Proceedings of the 10th International Symposium on Unmanned Untethered Submersible Technology, Durham, NH, USA, pp 406–415
Alvarez A, Caiti A (2002) Interactions of autonomous underwater vehicles with variable scale ocean structures. In: Proceedings of the IFAC World Conference Systems, Barcelona, Spain
Alvarez A, Caiti A, Onken R (2004) Evolutionary path planning for autonomous underwater vehicles in a variable ocean. IEEE J Ocean Eng 29(2):418–429
Garau B, Alvarez A, Oliver G (2005) Path planning of autonomous underwater vehicles in current fields with complex spatial variability: an A* approach. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, Barcelona, Spain, April 18–22, pp 194–198
Lee T, Chung H, Myung H (2011) Multi-resolution path planning for marine surface vehicle considering environmental effects. In: 2011 IEEE OCEANS, Santander, Spain, June 06–09, pp 1–9
Kim H, Lee T, Chung H, Son N, Myung H (2012) Any-angle path planning with limit-cycle circle set for marine surface vehicle. In: 2012 IEEE International Conference on Robotics and Automation, Saint Paul, MN, May 14–18, pp 2275–2280
Leyland G, Philpott A (2005) Rowing to Barbados. Dagstuhl Seminar Proceedings 05031 Algorithms for Optimization with Incomplete Information. http://drops.dagstuhl.de/opus/volltexte/2005/65
Shao W, Zhou P, Thong SK (2012) Development of a novel forward dynamic programming method for weather routing. J Mar Sci Technol 17(2):239–251
Lin YH, Fang MC, Yeung RW (2013) The optimization of ship weather-routing algorithm based the composite influence of multi-dynamic elements. Appl Ocean Res 43:184–194
Padhy CP, Sen D, Bhaskaran PK (2008) Application of wave model for weather routing of ships in the North Indian Ocean. Nat Hazards 44(3):373–385
Rao D, Williams SB (2009) Large-scale path planning for underwater gliders in ocean currents. In: Proceedings of 2009 Australasian Conference on Robotics and Automation, Sydney, Australia, December 02–04
Ross S, Chaib-draa B, Pineau J (2008) Bayesian reinforcement learning in continuous POMDPs with application to robot navigation. In: 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, May 19–23, pp 2845–2851
Fjerdingen SA, Kyrkjeboe E, Transeth AA (2010) AUV Pipeline Following using Reinforcement learning. In: 2010 41st International Symposium on and 2010 6th German Conference on Robotics, Munich, Germany, June 07–09, pp 1–8
El-Fakdi A, Carreras M (2013) Two-step gradient-based reinforcement learning for underwater robotics behavior learning. Robot Auton Syst 61(3):271–282
Carreres M, Yuh Junku, Batlle J, Ridao P (2005) A behavior-based scheme using reinforcement learning for autonomous underwater vehicles. IEEE J Ocean Eng 30(2):416–427
Laumond JP, Sekhavat S, Lamiraux F (1998) Robot Motion Planning and Control. Guidelines in nonholonomic motion planning for mobile robots. Springer, Berlin
Zexiang L, Canny JF (2012) Nonholonomic Motion Planning. Springer Science & Business Media
Chapra SC (2010) Numerical Methods for Engineers, 6th edn. McGraw-Hill
Karaman S, Frazzoli E (2011) Sampling-based algorithms for optimal motion planning. Int J Robot Res 30(7):846–894
Acknowledgments
This research was a part of the project titled ‘Development of Management Technology for HNS Accident', funded by the Ministry of Oceans and Fisheries, Korea. This research was also supported by a grant from the Hanwha Corporation, R&D Center, Korea.
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Yoo, B., Kim, J. Path optimization for marine vehicles in ocean currents using reinforcement learning. J Mar Sci Technol 21, 334–343 (2016). https://doi.org/10.1007/s00773-015-0355-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00773-015-0355-9