SpringerLink
Log in

Motion planning and control for mobile robot navigation using machine learning: a survey

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Moving in complex environments is an essential capability of intelligent mobile robots. Decades of research and engineering have been dedicated to develo** sophisticated navigation systems to move mobile robots from one point to another. Despite their overall success, a recently emerging research thrust is devoted to develo** machine learning techniques to address the same problem, based in large part on the success of deep learning. However, to date, there has not been much direct comparison between the classical and emerging paradigms to this problem. In this article, we survey recent works that apply machine learning for motion planning and control in mobile robot navigation, within the context of classical navigation systems. The surveyed works are classified into different categories, which delineate the relationship of the learning approaches to classical methods. Based on this classification, we identify common challenges and promising future directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. In mobile robot navigation, “motion planning” mostly focuses on relatively long-term sequences of robot positions, orientations, and their high-order derivatives, while motion control generally refers to relatively low-level motor commands, e.g., linear and angular velocities. However, the line between them is blurry, and we do not adhere to any strict distinction in terminology in this survey.

References

  • Becker-Ehmck, P., Karl, M., Peters, J., & van der Smagt, P. (2020). Learning to fly via deep model-based reinforcement learning. ar**v preprint ar**v:2003.08876

  • Bhardwaj, M., Boots, B., & Mukadam, M. (2020). Differentiable Gaussian process motion planning. In 2020 IEEE international conference on robotics and automation (ICRA) (pp. 10598–10604). IEEE.

  • Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016). End to end learning for self-driving cars. ar**v preprint ar**v:1604.07316

  • Bruce, J., Sünderhauf, N., Mirowski, P., Hadsell, R., & Milford, M. (2017). One-shot reinforcement learning for robot navigation with interactive replay. ar**v preprint ar**v:1711.10137

  • Chen, C., Liu, Y., Kreiss, S., & Alahi, A. (2019). Crowd–robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning. In 2019 international conference on robotics and automation (ICRA) (pp. 6015–6022). IEEE.

  • Chen, Y. F., Everett, M., Liu, M., & How, J. P. (2017). Socially aware motion planning with deep reinforcement learning. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 1343–1350). IEEE.

  • Chen, Y. F., Liu, M., Everett, M., & How, J. P. (2017). Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 285–292). IEEE

  • Chiang, H. T. L., Faust, A., Fiser, M., & Francis, A. (2019). Learning navigation behaviors end-to-end with autorl. IEEE Robotics and Automation Letters, 4(2), 2007–2014.

    Article  Google Scholar 

  • Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. ar**v preprint ar**v:1412.3555

  • Codevilla, F., Miiller, M., López, A., Koltun, V., & Dosovitskiy, A. (2018). End-to-end driving via conditional imitation learning. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 1–9). IEEE.

  • Daniel, K., Nash, A., Koenig, S., & Felner, A. (2010). Theta*: Any-angle path planning on grids. Journal of Artificial Intelligence Research, 39, 533–579.

    Article  MathSciNet  Google Scholar 

  • Dennis, M., Jaques, N., Vinitsky, E., Bayen, A., Russell, S., Critch, A., & Levine, S. (2020). Emergent complexity and zero-shot transfer via unsupervised environment design. In Advances in neural information processing systems (Vol. 33, pp. 13049–13061). Curran Associates, Inc.

  • Dijkstra, E. W. (1959). A note on two problems in connexion with graphs. Numerische Mathematik, 1(1), 269–271.

    Article  MathSciNet  Google Scholar 

  • Ding, W., Li, S., Qian, H., & Chen, Y. (2018). Hierarchical reinforcement learning framework towards multi-agent navigation. In 2018 IEEE international conference on robotics and biomimetics (ROBIO) (pp. 237–242). IEEE.

  • Durrant-Whyte, H., & Bailey, T. (2006). Simultaneous localization and map**: Part I. IEEE Robotics & Automation Magazine, 13(2), 99–110.

    Article  Google Scholar 

  • Elfes, A. (1989). Using occupancy grids for mobile robot perception and navigation. Computer, 22(6), 46–57.

    Article  Google Scholar 

  • Everett, M., Chen, Y. F., & How, J. P. (2018). Motion planning among dynamic, decision-making agents with deep reinforcement learning. In 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3052–3059). IEEE.

  • Faust, A., Oslund, K., Ramirez, O., Francis, A., Tapia, L., Fiser, M., & Davidson, J. (2018). Prm-rl: Long-range robotic navigation tasks by combining reinforcement learning and sampling-based planning. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 5113–5120). IEEE.

  • Fox, D., Burgard, W., & Thrun, S. (1997). The dynamic window approach to collision avoidance. IEEE Robotics & Automation Magazine, 4(1), 23–33.

    Article  Google Scholar 

  • Gao, W., Hsu, D., Lee, W. S., Shen, S., & Subramanian, K. (2017). Intention-net: Integrating planning and deep learning for goal-directed autonomous navigation. In Conference on robot learning (pp. 185–194). PMLR.

  • Giusti, A., Guzzi, J., Cireşan, D. C., He, F. L., Rodríguez, J. P., Fontana, F., et al. (2015). A machine learning approach to visual perception of forest trails for mobile robots. IEEE Robotics and Automation Letters, 1(2), 661–667.

    Article  Google Scholar 

  • Godoy, J., Chen, T., Guy, S. J., Karamouzas, I., & Gini, M. (2018). ALAN: Adaptive learning for multi-agent navigation. Autonomous Robots, 42(8), 1543–1562.

    Article  Google Scholar 

  • Gupta, S., Davidson, J., Levine, S., Sukthankar, R., & Malik, J. (2017) Cognitive map** and planning for visual navigation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2616–2625).

  • Gupta, S., Fouhey, D., Levine, S., & Malik, J. (2017). Unifying map and landmark based representations for visual navigation. ar**v preprint ar**v:1712.08125

  • Hart, P., Nilsson, N., & Raphael, B. (1968). A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics, 4(2), 100–107. https://doi.org/10.1109/tssc.1968.300136.

    Article  Google Scholar 

  • Henry, P., Vollmer, C., Ferris, B., & Fox, D. (2010). Learning to navigate through crowded environments. In 2010 IEEE international conference on robotics and automation (pp. 981–986). IEEE.

  • Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.

    Article  Google Scholar 

  • Jaillet, L., Cortés, J., & Siméon, T. (2010). Sampling-based path planning on configuration-space costmaps. IEEE Transactions on Robotics, 26(4), 635–646.

    Article  Google Scholar 

  • Jiang, P., Osteen, P., Wigness, M., & Saripalli, S. (2021). Rellis-3d dataset: Data, benchmarks and analysis. In 2021 IEEE international conference on robotics and automation (ICRA) (pp. 1110–1116). IEEE.

  • **, J., Nguyen, N. M., Sakib, N., Graves, D., Yao, H., & Jagersand, M. (2020). Mapless navigation among dynamics with social-safety-awareness: A reinforcement learning approach from 2d laser scans. In 2020 IEEE international conference on robotics and automation (ICRA) (pp. 6979–6985). IEEE.

  • Johnson, C., & Kuipers, B. (2018). Socially-aware navigation using topological maps and social norm learning. In Proceedings of the 2018 AAAI/ACM conference on AI, ethics, and society (pp. 151–157).

  • Kahn, G., Abbeel, P., & Levine, S. (2021). Badgr: An autonomous self-supervised learning-based navigation system. IEEE Robotics and Automation Letters, 6(2), 1312–1319.

    Article  Google Scholar 

  • Kahn, G., Villaflor, A., Abbeel, P., & Levine, S. (2018) Composable action-conditioned predictors: Flexible off-policy learning for robot navigation. In Conference on robot learning (pp. 806–816). PMLR.

  • Kahn, G., Villaflor, A., Ding, B., Abbeel, P., & Levine, S. (2018). Self-supervised deep reinforcement learning with generalized computation graphs for robot navigation. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 1–8). IEEE.

  • Karaman, S., & Frazzoli, E. (2011). Sampling-based algorithms for optimal motion planning. The International Journal of Robotics Research, 30(7), 846–894.

    Article  Google Scholar 

  • Kavraki, L. E., Svestka, P., Latombe, J. C., & Overmars, M. H. (1996). Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Transactions on Robotics and Automation, 12(4), 566–580.

    Article  Google Scholar 

  • Khan, A., Zhang, C., Atanasov, N., Karydis, K., Kumar, V., & Lee, D. D. (2018). Memory augmented control networks. In International conference on learning representations (ICLR).

  • Kim, B., & Pineau, J. (2016). Socially adaptive path planning in human environments using inverse reinforcement learning. International Journal of Social Robotics, 8(1), 51–66.

    Article  Google Scholar 

  • Koenig, S., & Likhachev, M. (2002). D\(\hat{\,}{}^{*}\) lite. In AAAI/IAAI (Vol. 15).

  • Kretzschmar, H., Spies, M., Sprunk, C., & Burgard, W. (2016). Socially compliant mobile robot navigation via inverse reinforcement learning. The International Journal of Robotics Research, 35(11), 1289–1307.

    Article  Google Scholar 

  • Kroemer, O., Niekum, S., & Konidaris, G. (2021). A review of robot learning for manipulation: Challenges, representations, and algorithms. Journal of Machine Learning Research, 22, 30–1.

    MathSciNet  MATH  Google Scholar 

  • LaValle, S. M. (1998). Rapidly-exploring random trees: A new tool for path planning.

  • LaValle, S. M. (2006). Planning algorithms. Cambridge University Press.

  • LeCunn, Y., Muller, U., Ben, J., Cosatto, E., & Flepp, B. (2006). Off-road obstacle avoidance through end-to-end learning. In Advances in neural information processing systems (pp. 739–746).

  • Li, M., Jiang, R., Ge, S. S., & Lee, T. H. (2018). Role playing learning for socially concomitant mobile robot navigation. CAAI Transactions on Intelligence Technology, 3(1), 49–58.

    Article  Google Scholar 

  • Liang, J., Patel, U., Sathyamoorthy, A. J., & Manocha, D. (2020). Crowd-steer: Realtime smooth and collision-free robot navigation in densely crowded scenarios trained using high-fidelity simulation. In IJCAI (pp. 4221–4228).

  • Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015). Continuous control with deep reinforcement learning. ar**v preprint ar**v:1509.02971

  • Lin, J., Wang, L., Gao, F., Shen, S., & Zhang, F. (2019). Flying through a narrow gap using neural network: An end-to-end planning and control approach. In 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3526–3533). IEEE.

  • Liu, B., **ao, X., & Stone, P. (2021). A lifelong learning approach to mobile robot navigation. IEEE Robotics and Automation Letters, 6(2), 1090–1096.

    Article  Google Scholar 

  • Long, P., Fanl, T., Liao, X., Liu, W., Zhang, H., & Pan, J. (2018). Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 6252–6259). IEEE.

  • Lopez-Paz, D., & Ranzato, M. (2017). Gradient episodic memory for continual learning. In Advances in neural information processing systems (pp. 6467–6476).

  • Loquercio, A., Maqueda, A. I., Del-Blanco, C. R., & Scaramuzza, D. (2018). Dronet: Learning to fly by driving. IEEE Robotics and Automation Letters, 3(2), 1088–1095.

    Article  Google Scholar 

  • Lu, D. V., Hershberger, D., & Smart, W. D. (2014). Layered costmaps for context-sensitive navigation. In 2014 IEEE/RSJ international conference on intelligent robots and systems (pp. 709–715). IEEE.

  • Luber, M., Spinello, L., Silva, J., & Arras, K. O. (2012). Socially-aware robot navigation: A learning approach. In 2012 IEEE/RSJ international conference on intelligent robots and systems (pp. 902–907). IEEE.

  • Martins, G. S., Rocha, R. P., Pais, F. J., & Menezes, P. (2019). Clusternav: Learning-based robust navigation operating in cluttered environments. In 2019 international conference on robotics and automation (ICRA) (pp. 9624–9630). IEEE.

  • Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., & Kavukcuoglu, K. (2016). Asynchronous methods for deep reinforcement learning. In International conference on machine learning (pp. 1928–1937).

  • Nistér, D., Naroditsky, O., & Bergen, J. (2004). Visual odometry. In Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004 (Vol. 1, p. I). IEEE.

  • Okal, B., & Arras, K. O. (2016). Learning socially normative robot navigation behaviors with Bayesian inverse reinforcement learning. In 2016 IEEE international conference on robotics and automation (ICRA) (pp. 2889–2895). IEEE.

  • OSRF. (2018). Ros wiki move_base. http://wiki.ros.org/move_base

  • Palmieri, L., & Arras, K. O. (2014). Efficient and smooth RRT motion planning using a novel extend function for wheeled mobile robots. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 205–211).

  • Pan, Y., Cheng, C. A., Saigol, K., Lee, K., Yan, X., Theodorou, E. A., & Boots, B. (2020). Imitation learning for agile autonomous driving. The International Journal of Robotics Research, 39(2–3), 286–302.

    Article  Google Scholar 

  • Park, J. J. (2016). Graceful navigation for mobile robots in dynamic and uncertain environments. Ph.D. thesis.

  • Pérez-Higueras, N., Caballero, F., & Merino, L. (2018). Learning human-aware path planning with fully convolutional networks. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 1–5). IEEE.

  • Pérez-Higueras, N., Caballero, F., & Merino, L. (2018). Teaching robot navigation behaviors to optimal RRT planners. International Journal of Social Robotics, 10(2), 235–249.

    Article  Google Scholar 

  • Pfeiffer, M., Schaeuble, M., Nieto, J., Siegwart, R., & Cadena, C. (2017). From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 1527–1533). IEEE.

  • Pfeiffer, M., Schwesinger, U., Sommer, H., Galceran, E., & Siegwart, R. (2016). Predicting actions to act predictably: Cooperative partial motion planning with maximum entropy models. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2096–2101). IEEE.

  • Pfeiffer, M., Shukla, S., Turchetta, M., Cadena, C., Krause, A., Siegwart, R., & Nieto, J. (2018). Reinforced imitation: Sample efficient deep reinforcement learning for mapless navigation by leveraging prior demonstrations. IEEE Robotics and Automation Letters, 3(4), 4423–4430.

    Article  Google Scholar 

  • Pokle, A., Martín-Martín, R., Goebel, P., Chow, V., Ewald, H. M., Yang, J., Wang, Z., Sadeghian, A., Sadigh, D., Savarese, S.,et al. (2019). Deep local trajectory replanning and control for robot navigation. In 2019 international conference on robotics and automation (ICRA) (pp. 5815–5822). IEEE.

  • Pomerleau, D. A. (1989). Alvinn: An autonomous land vehicle in a neural network. In Advances in neural information processing systems (pp. 305–313).

  • Quinlan, S., & Khatib, O. (1993). Elastic bands: Connecting path planning and control. In [1993] Proceedings IEEE international conference on robotics and automation (pp. 802–807). IEEE.

  • Ramachandran, D., & Amir, E. (2007). Bayesian inverse reinforcement learning. In IJCAI (Vol. 7, pp. 2586–2591).

  • Richter, C., & Roy, N. (2017). Safe visual navigation via deep learning and novelty detection. In Robotics: Science and systems (RSS).

  • Ross, S., Gordon, G., & Bagnell, D. (2011). A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics (pp. 627–635).

  • Ross, S., Melik-Barkhudarov, N., Shankar, K. S., Wendel, A., Dey, D., Bagnell, J. A., & Hebert, M. (2013). Learning monocular reactive UAV control in cluttered natural environments. In 2013 IEEE international conference on robotics and automation (pp. 1765–1772). IEEE.

  • Russell, S. J., & Norvig, P. (2016). Artificial intelligence: A modern approach. Pearson Education Limited.

  • Sadeghi, F., & Levine, S. (2017). CAD2RL: Real single-image flight without a single real image. In Robotics: Science and systems (RSS).

  • Sepulveda, G., Niebles, J. C., & Soto, A. (2018). A deep learning based behavioral approach to indoor autonomous navigation. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 4646–4653). IEEE.

  • Sergeant, J., Sünderhauf, N., Milford, M., & Upcroft, B. (2015). Multimodal deep autoencoders for control of a mobile robot. In Proceedings of Australasian conference for robotics and automation (ACRA).

  • Shiarlis, K., Messias, J., & Whiteson, S. (2017). Rapidly exploring learning trees. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 1541–1548). IEEE.

  • Siva, S., Wigness, M., Rogers, J., & Zhang, H. (2019). Robot adaptation to unstructured terrains by joint representation and apprenticeship learning. In Robotics: Science and systems (RSS).

  • Sood, R., Vats, S., & Likhachev, M. (2020). Learning to use adaptive motion primitives in search-based planning for navigation. In 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 6923–6929). IEEE.

  • Stein, G. J., Bradley, C., & Roy, N. (2018). Learning over subgoals for efficient navigation of structured, unknown environments. In Conference on robot learning (pp. 213–222).

  • Stratonovich, R. L. (1965). Conditional Markov processes. In Non-linear transformations of stochastic processes (pp. 427–453). Elsevier.

  • Tai, L., Li, S., & Liu, M. (2016). A deep-network solution towards model-less obstacle avoidance. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2759–2764). IEEE.

  • Tai, L., & Liu, M. (2016). Deep-learning in mobile robotics-from perception to control systems: A survey on why and why not. ar**v preprint ar**v:1612.07139

  • Tai, L., Paolo, G., & Liu, M. (2017). Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 31–36). IEEE.

  • Tai, L., Zhang, J., Liu, M., Boedecker, J., & Burgard, W. (2016). A survey of deep network solutions for learning control in robotics: From reinforcement to imitation. ar**v preprint ar**v:1612.07139

  • Tai, L., Zhang, J., Liu, M., & Burgard, W. (2018). Socially compliant navigation through raw depth inputs with generative adversarial imitation learning. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 1111–1117). IEEE.

  • Tamar, A., Wu, Y., Thomas, G., Levine, S., & Abbeel, P. (2016). Value iteration networks. In Advances in neural information processing systems (pp. 2154–2162).

  • Teso-Fz-Betoño, D., Zulueta, E., Fernandez-Gamiz, U., Saenz-Aguirre, A., & Martinez, R. (2019). Predictive dynamic window approach development with artificial neural fuzzy inference improvement. Electronics, 8(9), 935.

    Article  Google Scholar 

  • Thrun, S. (1995). An approach to learning mobile robot navigation. Robotics and Autonomous Systems, 15(4), 301–319.

    Article  Google Scholar 

  • Ullman, S. (1979). The interpretation of structure from motion. Proceedings of the Royal Society of London. Series B. Biological Sciences, 203(1153), 405–426.

    Google Scholar 

  • Van Den Berg, J., Guy, S. J., Lin, M., & Manocha, D. (2011). Reciprocal n-body collision avoidance. In Robotics research (pp. 3–19). Springer.

  • Wang, Y., He, H., & Sun, C. (2018). Learning to navigate through complex dynamic environment with modular deep reinforcement learning. IEEE Transactions on Games, 10(4), 400–412.

    Article  Google Scholar 

  • Wang, Z., **ao, X., Liu, B., Warnell, G., & Stone, P. (2021). Appli: Adaptive planner parameter learning from interventions. In 2021 IEEE international conference on robotics and automation (ICRA) (pp. 6079–6085). IEEE.

  • Wang, Z., **ao, X., Nettekoven, A. J., Umasankar, K., Singh, A., Bommakanti, S., Topcu, U., & Stone, P. (2021). From agile ground to aerial navigation: Learning from learned hallucination. In 2021 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE.

  • Wang, Z., **ao, X., Warnell, G., & Stone, P. (2021). Apple: Adaptive planner parameter learning from evaluative feedback. IEEE Robotics and Automation Letters, 6(4), 7744–7749.

    Article  Google Scholar 

  • Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine Learning, 8(3–4), 279–292.

    MATH  Google Scholar 

  • Wigness, M., Rogers, J. G., & Navarro-Serment, L. E. (2018). Robot navigation from human demonstration: Learning control behaviors. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 1150–1157). IEEE.

  • **ao, X., Biswas, J., & Stone, P. (2021a). Learning inverse kinodynamics for accurate high-speed off-road navigation on unstructured terrain. IEEE Robotics and Automation Letters, 6(3), 6054–6060.

  • **ao, X., Liu, B., & Stone, P. (2021b). Agile robot navigation through hallucinated learning and sober deployment. In 2021 IEEE international conference on robotics and automation (ICRA) (pp. 7316–7322). IEEE.

  • **ao, X., Liu, B., Warnell, G., Fink, J., & Stone, P. (2020). Appld: Adaptive planner parameter learning from demonstration. IEEE Robotics and Automation Letters, 5(3), 4541–4547.

    Article  Google Scholar 

  • **ao, X., Liu, B., Warnell, G., & Stone, P. (2021c). Toward agile maneuvers in highly constrained spaces: Learning from hallucination. IEEE Robotics and Automation Letters, 6(2), 1503–1510.

    Article  Google Scholar 

  • **ao, X., Wang, Z., Xu, Z., Liu, B., Warnell, G., Dhamankar, G., Nair, A., & Stone, P. (2021d). Appl: Adaptive planner parameter learning. ar**v preprint ar**v:2105.07620

  • **e, L., Wang, S., Rosa, S., Markham, A., & Trigoni, N. (2018). Learning with training wheels: Speeding up training with a simple controller for deep reinforcement learning. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 6276–6283). IEEE.

  • Xu, Z., Dhamankar, G., Nair, A., **ao, X., Warnell, G., Liu, B., Wang, Z., & Stone, P. (2021). Applr: Adaptive planner parameter learning from reinforcement. In 2021 IEEE international conference on robotics and automation (ICRA) (pp. 6086–6092). IEEE.

  • Yao, X., Zhang, J., & Oh, J. (2019). Following social groups: Socially compliant autonomous navigation in dense crowds. ar**v preprint ar**v:1911.12063

  • Zeng, J., Ju, R., Qin, L., Hu, Y., Yin, Q., & Hu, C. (2019). Navigation in unknown dynamic environments based on deep reinforcement learning. Sensors, 19(18), 3837.

    Article  Google Scholar 

  • Zhang, J., Springenberg, J. T., Boedecker, J., & Burgard, W. (2017). Deep reinforcement learning with successor features for navigation across similar environments. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2371–2378). IEEE.

  • Zhang, J., Tai, L., Boedecker, J., Burgard, W., & Liu, M. (2017). Neural slam: Learning to explore with external memory. ar**v preprint ar**v:1706.09520

  • Zhang, T., Kahn, G., Levine, S., & Abbeel, P. (2016). Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search. In 2016 IEEE international conference on robotics and automation (ICRA) (pp. 528–535). IEEE.

  • Zhao, L., & Roh, M. I. (2019). Colregs-compliant multiship collision avoidance based on deep reinforcement learning. Ocean Engineering, 191, 106436.

    Article  Google Scholar 

  • Zhelo, O., Zhang, J., Tai, L., Liu, M., & Burgard, W. (2018). Curiosity-driven exploration for mapless navigation with deep reinforcement learning. ar**v preprint ar**v:1804.00456

  • Zhou, X., Gao, Y., & Guan, L. (2019). Towards goal-directed navigation through combining learning based global and local planners. Sensors, 19(1), 176.

    Article  Google Scholar 

  • Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017). Target-driven visual navigation in indoor scenes using deep reinforcement learning. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 3357–3364). IEEE.

  • Zhu, Y., Schwab, D., & Veloso, M. (2019). Learning primitive skills for mobile robots. In 2019 international conference on robotics and automation (ICRA) (pp. 7597–7603). IEEE.

  • Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2008). Maximum entropy inverse reinforcement learning. In AAAI (Vol. 8, pp. 1433–1438).

Download references

Acknowledgements

This work has taken place in the Learning Agents Research Group (LARG) at the Artificial Intelligence Laboratory, The University of Texas at Austin. LARG research is supported in part by Grants from the National Science Foundation (CPS-1739964, IIS-1724157, NRI-1925082), the Office of Naval Research (N00014-18-2243), Future of Life Institute (RFP2-000), Army Research Office (W911NF-19-2-0333), DARPA, Lockheed Martin, General Motors, and Bosch. The views and conclusions contained in this document are those of the authors alone. Peter Stone serves as the Executive Director of Sony AI America and receives financial compensation for this work. The terms of this arrangement have been reviewed and approved by the University of Texas at Austin in accordance with its policy on objectivity in research. We would also like to thank Yifeng Zhu for helpful discussions and suggestions, and Siddharth Rajesh Desai for hel** editing and refining the language for this survey.

Author information

Authors and Affiliations

  1. Department of Computer Science, The University of Texas at Austin, Austin, TX, 78712, USA

    Xuesu **ao, Bo Liu & Peter Stone

  2. Sony AI, Tokyo, Japan

    Peter Stone

  3. Computational and Information Sciences Directorate, Army Research Laboratory, Adelphi, MD, 20783, USA

    Garrett Warnell

Authors
  1. Xuesu **ao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Garrett Warnell
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Stone
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuesu **ao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

**ao, X., Liu, B., Warnell, G. et al. Motion planning and control for mobile robot navigation using machine learning: a survey. Auton Robot 46, 569–597 (2022). https://doi.org/10.1007/s10514-022-10039-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-022-10039-8

Keywords

Search

Navigation