SpringerLink
Log in

Machine Learning for Unmanned Aerial Vehicles Navigation: An Overview

  • Survey Article
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Unmanned aerial vehicles (UAVs) are a valuable source of data for a wide range of real-time applications, due to their functionality, availability, adaptability, and maneuverability. Working as mobile sensors, they can provide a cost-effective solution for extremely complex tasks, such as inspection, air-to-ground communications, search and rescue, surveillance, among others. Nevertheless, the robots needs to navigate in quite distinct environments and in different dynamism levels, usually facing unpredicted situations, very often using limited sensing and computing capabilities. A large number of solutions to this problem has been featured by the scientific community in the last years, some of them based on machine-learning (ML) methods. Due to its great capability to deal with big data and complexity, as well as its speedy and high-accuracy processing, the ML framework has been used to improve existing technologies and control techniques. In this context, its adoption in several UAV navigation strategies is expected to provide solutions for various problems where UAVs are used in real-time applications. Thus, in order to contextualize the most recent advances, this work provides a detailed survey of relevant researches in which ML techniques have been used in UAV navigation to improve some functional aspects, such as energy-efficiency, communication, execution time, resource management, obstacle avoidance, and path planning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The datasets analyzed during the current work are available from the corresponding author on reasonable request.

Notes

  1. GPS signals could be weak or totally lost in scenarios like urban areas, low altitude operations or indoor flights, for instance.

Abbreviations

A3C:

Asynchronous advantage actor-critic

AI:

Artificial intelligence

ANN:

Artificial Neural Network

BIF:

Bayesian information filter

CNN:

Convolutional Neural Network

DDPGfD:

Deep deterministic policy gradient from demonstrations

DDPG:

Deep deterministic policy gradient

DL:

Deep learning

DRL:

Deep reinforcement learning

EKF:

Extend Kalman filter

ESC:

Electronic speed controllers

GCS:

Ground control station

GNSS:

Global navigation satellite system

GPS:

Global position system

HiL:

Hardware in the loop

HRI:

Human–robot interface

IMU:

Inertial measurement unit

INS:

Inertial navigation system

LiDAR:

Light detection and ranging

LSTM:

Long Short-Term Memory

LoS:

Line-of-sight

MDP:

Markov decision process

ML:

Machine learning

MLP:

Multilayer perceptrons

NS:

Navigation system

OGV:

On-ground vision

PHY:

Physical layer

POfD:

Policy optimization from demonstrations

PWM:

Pulse width modulation

RF:

Radio-frequency

RL:

Reinforcement learning

SA:

Situational awareness

SL:

Supervised learning

SLAM:

Simultaneous localization and map**

UAS:

Unmanned aerial systems

UAV:

Unmanned aerial vehicles

UL:

Unsupervised learning

VO:

Visual odometry

References

  1. Mohamed N, Al-Jaroodi J, Jawhar I, Idries A, Mohammed F. Unmanned aerial vehicles applications in future smart cities. Technol Forecast Soc Change. 2020;153:119293.

    Article  Google Scholar 

  2. Doughty CL, Cavanaugh KC. Map** coastal wetland biomass from high resolution unmanned aerial vehicle (UAV) imagery. Remote Sens. 2019;11(5):540.

    Article  ADS  Google Scholar 

  3. Osco LP, Junior JM, Ramos APM, de Castro Jorge LA, Fatholahi SN, de Andrade Silva J, Matsubara ET, Pistori H, Gonçalves WN, Li J. A review on deep learning in UAV remote sensing. Int J Appl Earth Obs Geoinf. 2021;102:102456.

    Google Scholar 

  4. Sun W, Dai L, Zhang X, Chang P, He X. RSOD: real-time small object detection algorithm in UAV-based traffic monitoring. Appl Intell. 2021;52:8448–63.

    Article  Google Scholar 

  5. Mohiuddin A, Taha T, Zweiri Y, Gan D. Dual-UAV payload transportation using optimized velocity profiles via real-time dynamic programming. Drones. 2023;7(3):171.

    Article  Google Scholar 

  6. López S, Cervantes J-A, Cervantes S, Molina J, Cervantes F. The plausibility of using unmanned aerial vehicles as a serious game for dealing with attention deficit-hyperactivity disorder. Cogn Syst Res. 2020;59:160–70.

    Article  Google Scholar 

  7. Sethuraman SC, Tadkapally GR, Mohanty SP, Subramanian A. iDRONE: IoT-enabled unmanned aerial vehicles for detecting wildfires using convolutional neural networks. SN Comput Sci. 2022;3(3):242.

    Article  Google Scholar 

  8. Dabas C. Insight of unmanned aerial vehicles accessing ensemble techniques. SN Comput Sci. 2021;2(6):458.

    Article  Google Scholar 

  9. Balamurugan G, Valarmathi J, Naidu V. Survey on UAV navigation in GPS denied environments. In: 2016 International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES). IEEE; 2016. p. 198–204.

  10. Yang G-Z, Bellingham J, Dupont PE, Fischer P, Floridi L, Full R, Jacobstein N, Kumar V, McNutt M, Merrifield R. The grand challenges of science robotics. Sci Robot. 2018;3(14):7650.

    Article  Google Scholar 

  11. Lu Y, Xue Z, **a G-S, Zhang L. A survey on vision-based UAV navigation. Geo-spat Inf Sci. 2018;21(1):21–32.

    Article  Google Scholar 

  12. Queralta JP, Almansa CM, Schiano F, Floreano D, Westerlund T. UWB-based system for UAV localization in GNSS-denied environments: characterization and dataset (2020). ar**v preprint ar**v:2003.04380.

  13. Gupte S, Mohandas PIT, Conrad JM. A survey of quadrotor unmanned aerial vehicles. In: 2012 Proceedings of IEEE Southeastcon. IEEE; 2012. p. 1–6.

  14. Kendoul F. Survey of advances in guidance, navigation, and control of unmanned rotorcraft systems. J Field Robot. 2012;29(2):315–78.

    Article  Google Scholar 

  15. Carrio A, Sampedro C, Rodriguez-Ramos A, Campoy P. A review of deep learning methods and applications for unmanned aerial vehicles. J Sens. 2017;2017(2):1–13.

    Article  Google Scholar 

  16. Zahran S, Moussa A, El-Sheimy N. Enhanced UAV navigation in GNSS denied environment using repeated dynamics pattern recognition. In: 2018 IEEE/ION Position, Location and Navigation Symposium (PLANS). IEEE; 2018. p. 1135–42.

  17. Virnodkar SS, Pachghare VK, Patil V, Jha SK. Remote sensing and machine learning for crop water stress determination in various crops: a critical review. Precis Agric. 2020;21(5):1121–55.

    Article  Google Scholar 

  18. Hung GL, Sahimi MSB, Samma H, Almohamad TA, Lahasan B. Faster R-CNN deep learning model for pedestrian detection from drone images. SN Comput Sci. 2020;1:1–9.

    Article  Google Scholar 

  19. Chriki A, Touati H, Snoussi H, Kamoun F. Centralized cognitive radio based frequency allocation for UAVs communication. In: 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC). IEEE; 2019. p. 1674–9.

  20. Iqbal S. A study on UAV operating system security and future research challenges. In: 2021 IEEE 11th Annual Computing and Communication Workshop and Conference (CCWC). IEEE; 2021. p. 0759–65.

  21. Touati H, Chriki A, Snoussi H, Kamoun F. Cognitive radio and dynamic TDMA for efficient UAVs swarm communications. Comput Netw. 2021;196:108264.

    Article  Google Scholar 

  22. Khan NA, Jhanjhi NZ, Brohi SN, Nayyar A. Emerging use of UAV’s: secure communication protocol issues and challenges. In: Drones in smart-cities. Amsterdam: Elsevier; 2020. p. 37–55.

    Chapter  Google Scholar 

  23. Khan NA, Brohi SN, Jhanjhi N. UAV’s applications, architecture, security issues and attack scenarios: a survey. In: Intelligent computing and innovation on data science. Berlin: Springer; 2020. p. 753–60.

    Chapter  Google Scholar 

  24. Choi SY, Cha D. Unmanned aerial vehicles using machine learning for autonomous flight; state-of-the-art. Adv Robot. 2019;33(6):265–77.

    Article  Google Scholar 

  25. Bithas PS, Michailidis ET, Nomikos N, Vouyioukas D, Kanatas AG. A survey on machine-learning techniques for UAV-based communications. Sensors. 2019;19(23):5170.

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  26. Mitchell R, Michalski J, Carbonell T. An artificial intelligence approach. Berlin: Springer; 2013.

    Google Scholar 

  27. Biamonte J, Wittek P, Pancotti N, Rebentrost P, Wiebe N, Lloyd S. Quantum machine learning. Nature. 2017;549(7671):195–202.

    Article  ADS  CAS  PubMed  Google Scholar 

  28. Alpaydin E, Cambridge. Introduction to machine learning. MIT Press; 2020.

    Google Scholar 

  29. Yi**g Z, Zheng Z, **aoyi Z, Yang L. Q learning algorithm based UAV path learning and obstacle avoidence approach. In: 2017 36th Chinese Control Conference (CCC). IEEE; 2017. p. 3397–402.

  30. Yavanoglu O, Aydos M. A review on cyber security datasets for machine learning algorithms. In: 2017 IEEE International Conference on Big Data (big Data). IEEE; 2017. p. 2186–93.

  31. Hu W, Fey M, Zitnik M, Dong Y, Ren H, Liu B, Catasta M, Leskovec J. Open graph benchmark: datasets for machine learning on graphs. Adv Neural Inf Process Syst. 2020;33:22118–33.

    Google Scholar 

  32. Gauen K, Dailey R, Laiman J, Zi Y, Asokan N, Lu Y-H, Thiruvathukal GK, Shyu M-L, Chen S-C. Comparison of visual datasets for machine learning. In: 2017 IEEE International Conference on Information Reuse and Integration (IRI). IEEE; 2017. p. 346–55.

  33. Prakash KB, Imambi SS, Ismail M, Kumar TP, Pawan Y. Analysis, prediction and evaluation of Covid-19 datasets using machine learning algorithms. Int J. 2020;8(5).

  34. Kocheturov A, Pardalos PM, Karakitsiou A. Massive datasets and machine learning for computational biomedicine: trends and challenges. Ann Oper Res. 2019;276(1):5–34.

    Article  MathSciNet  Google Scholar 

  35. Challita U, Ferdowsi A, Chen M, Saad W. Machine learning for wireless connectivity and security of cellular-connected UAVs. IEEE Wirel Commun. 2019;26(1):28–35.

    Article  Google Scholar 

  36. Kouhdaragh V, Verde F, Gelli G, Abouei J. On the application of machine learning to the design of UAV-based 5G radio access networks. Electronics. 2020;9(4):689.

    Article  Google Scholar 

  37. Han L, Yang G, Dai H, Xu B, Yang H, Feng H, Li Z, Yang X. Modeling maize above-ground biomass based on machine learning approaches using UAV remote-sensing data. Plant Methods. 2019;15(1):1–19.

    Article  CAS  Google Scholar 

  38. Abdulridha J, Batuman O, Ampatzidis Y. UAV-based remote sensing technique to detect citrus canker disease utilizing hyperspectral imaging and machine learning. Remote Sens. 2019;11(11):1373.

    Article  ADS  Google Scholar 

  39. Abeysinghe T, Simic Milas A, Arend K, Hohman B, Reil P, Gregory A, Vázquez-Ortega A. Map** invasive phragmites australis in the old woman creek estuary using UAV remote sensing and machine learning classifiers. Remote Sens. 2019;11(11):1380.

    Article  ADS  Google Scholar 

  40. Lei L, Shen G, Zhang L, Li Z. Toward intelligent cooperation of UAV swarms: when machine learning meets digital twin. IEEE Netw. 2020;35(1):386–92.

    Article  Google Scholar 

  41. Wang X, Xu Y, Chen C, Yang X, Chen J, Ruan L, Xu Y, Chen R. Machine learning empowered spectrum sharing in intelligent unmanned swarm communication systems: challenges, requirements and solutions. IEEE Access. 2020;8:89839–49.

    Article  Google Scholar 

  42. Maimaitijiang M, Sagan V, Sidike P, Daloye AM, Erkbol H, Fritschi FB. Crop monitoring using satellite/UAV data fusion and machine learning. Remote Sens. 2020;12(9):1357.

    Article  ADS  Google Scholar 

  43. Ge X, Wang J, Ding J, Cao X, Zhang Z, Liu J, Li X. Combining UAV-based hyperspectral imagery and machine learning algorithms for soil moisture content monitoring. PeerJ. 2019;7:6926.

    Article  Google Scholar 

  44. Neuville R, Bates JS, Jonard F. Estimating forest structure from UAV-mounted lidar point cloud using machine learning. Remote Sens. 2021;13(3):352.

    Article  ADS  Google Scholar 

  45. Wang J-L, Li Y-R, Adege AB, Wang L-C, Jeng S-S, Chen J-Y. Machine learning based rapid 3d channel modeling for UAV communication networks. In: 2019 16th IEEE Annual Consumer Communications & Networking Conference (CCNC). IEEE; 2019. p. 1–5.

  46. Zhao Y, Zheng Z, Liu Y. Survey on computational-intelligence-based UAV path planning. Knowl-Based Syst. 2018;158:54–64.

    Article  Google Scholar 

  47. Kanellakis C, Nikolakopoulos G. Survey on computer vision for UAVs: current developments and trends. J Intell Robot Syst. 2017;87(1):141–68.

    Article  Google Scholar 

  48. Al-Kaff A, Martin D, Garcia F, de la Escalera A, Armingol JM. Survey of computer vision algorithms and applications for unmanned aerial vehicles. Expert Syst Appl. 2018;92:447–63.

    Article  Google Scholar 

  49. Sanchez-Rodriguez J-P, Aceves-Lopez A. A survey on stereo vision-based autonomous navigation for multi-rotor MUAVs. Robotica. 2018;36(8):1225–43.

    Article  Google Scholar 

  50. Luong NC, Hoang DT, Gong S, Niyato D, Wang P, Liang Y-C, Kim DI. Applications of deep reinforcement learning in communications and networking: a survey. IEEE Commun Surv Tutor. 2019;21(4):3133–74.

    Article  Google Scholar 

  51. Azar AT, Koubaa A, Ali Mohamed N, Ibrahim HA, Ibrahim ZF, Kazim M, Ammar A, Benjdira B, Khamis AM, Hameed IA. Drone deep reinforcement learning: a review. Electronics. 2021;10(9):999.

    Article  Google Scholar 

  52. Rezwan S, Choi W. Artificial intelligence approaches for UAV navigation: recent advances and future challenges. IEEE Access. 2022;10:26320–39.

    Article  Google Scholar 

  53. Alhafnawi M, Salameh HB, Masadeh A, Al-Obiedollah H, Ayyash M, El-Khazali R, Elgala H. A survey of indoor and outdoor UAV-based target tracking systems: current status, challenges, technologies, and future directions. IEEE Access. 2023;11:68324–39.

    Article  Google Scholar 

  54. McEnroe P, Wang S, Liyanage M. A survey on the convergence of edge computing and ai for UAVs: opportunities and challenges. IEEE Internet Things J. 2022;9(17):15435–59.

    Article  Google Scholar 

  55. Polydoros AS, Nalpantidis L. Survey of model-based reinforcement learning: applications on robotics. J Intell Robot Syst. 2017;86(2):153–73.

    Article  Google Scholar 

  56. Fraga-Lamas P, Ramos L, Mondéjar-Guerra V, Fernández-Caramés TM. A review on IoT deep learning UAV systems for autonomous obstacle detection and collision avoidance. Remote Sens. 2019;11(18):2144.

    Article  ADS  Google Scholar 

  57. Ullah H, Nair NG, Moore A, Nugent C, Muschamp P, Cuevas M. 5G communication: an overview of vehicle-to-everything, drones, and healthcare use-cases. IEEE Access. 2019;7:37251–68.

    Article  Google Scholar 

  58. Mittal P, Sharma A, Singh R. Deep learning-based object detection in low-altitude UAV datasets: a survey. Image Vis Comput. 2020;104:104046.

    Article  Google Scholar 

  59. Li B, Fei Z, Zhang Y. UAV communications for 5G and beyond: recent advances and future trends. IEEE Internet Things J. 2018;6(2):2241–63.

    Article  Google Scholar 

  60. Ullah Z, Al-Turjman F, Mostarda L. Cognition in UAV-aided 5G and beyond communications: a survey. IEEE Trans Cognit Commun Netw. 2020;6(3):872–91.

    Article  Google Scholar 

  61. Hu J, Zhang H, Song L, Han Z, Poor HV. Reinforcement learning for a cellular internet of UAVs: protocol design, trajectory control, and resource management. IEEE Wirel Commun. 2020;27(1):116–23.

    Article  CAS  Google Scholar 

  62. Lahmeri M-A, Kishk MA, Alouini M-S. Artificial intelligence for UAV-enabled wireless networks: a survey. IEEE Open J Commun Soc. 2021;2:1015–40.

    Article  Google Scholar 

  63. Srivastava S, Narayan S, Mittal S. A survey of deep learning techniques for vehicle detection from UAV images. J Syst Archit. 2021;117:102152.

    Article  Google Scholar 

  64. Olivares-Mendez MA, Fu C, Ludivig P, Bissyandé TF, Kannan S, Zurad M, Annaiyan A, Voos H, Campoy P. Towards an autonomous vision-based unmanned aerial system against wildlife poachers. Sensors. 2015;15(12):31362–91.

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  65. Nex F, Remondino F. UAV for 3D map** applications: a review. Appl Geomat. 2014;6(1):1–15.

    Article  Google Scholar 

  66. Baena S, Boyd DS, Moat J. UAVs in pursuit of plant conservation-real world experiences. Eco Inform. 2018;47:2–9.

    Article  Google Scholar 

  67. Ross S, Melik-Barkhudarov N, Shankar KS, Wendel A, Dey D, Bagnell JA, Hebert M. Learning monocular reactive UAV control in cluttered natural environments. In: 2013 IEEE International Conference on Robotics and Automation. IEEE; 2013. p. 1765–72.

  68. Pizetta IHB, Brandao AS, Sarcinelli-Filho M. Control and obstacle avoidance for an UAV carrying a load in forestal environments. In: 2018 International Conference on Unmanned Aircraft Systems (ICUAS). IEEE; 2018. p. 62–7.

  69. Sanfourche M, Delaune J, Le Besnerais G, De Plinval H, Israel J, Cornic P, Treil A, Watanabe Y, Plyer A. Perception for UAV: vision-based navigation and environment modeling. AerospaceLab. 2012;1(4):1.

    Google Scholar 

  70. Qin H, Meng Z, Meng W, Chen X, Sun H, Lin F, Ang MH. Autonomous exploration and map** system using heterogeneous UAVs and UGVs in GPS-denied environments. IEEE Trans Veh Technol. 2019;68(2):1339–50.

    Article  Google Scholar 

  71. Azevedo F, Dias A, Almeida J, Oliveira A, Ferreira A, Santos T, Martins A, Silva E. Lidar-based real-time detection and modeling of power lines for unmanned aerial vehicles. Sensors. 2019;19(8):1812.

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  72. Geraldes R, Goncalves A, Lai T, Villerabel M, Deng W, Salta A, Nakayama K, Matsuo Y, Prendinger H. UAV-based situational awareness system using deep learning. IEEE Access. 2019;7:122583–94.

    Article  Google Scholar 

  73. Goerzen C, Kong Z, Mettler B. A survey of motion planning algorithms from the perspective of autonomous UAV guidance. J Intell Rob Syst. 2010;57(1):65–100.

    Article  Google Scholar 

  74. Luebke D, Humphreys G. How GPUs work. Computer. 2007;40(2):96–100.

    Article  Google Scholar 

  75. Sarker IH. Machine learning: algorithms, real-world applications and research directions. SN Comput Sci. 2021;2(3):160.

    Article  PubMed  PubMed Central  Google Scholar 

  76. Yu Y, Si X, Hu C, Zhang J. A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput. 2019;31(7):1235–70.

    Article  MathSciNet  PubMed  Google Scholar 

  77. Škrlj B, Kralj J, Lavrač N, Pollak S. Towards robust text classification with semantics-aware recurrent neural architecture. Mach Learn Knowl Extr. 2019;1(2):575–89.

    Article  Google Scholar 

  78. Shyalika C, Silva T, Karunananda A. Reinforcement learning in dynamic task scheduling: a review. SN Comput Sci. 2020;1:1–17.

    Article  Google Scholar 

  79. Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J. Recent advances in convolutional neural networks. Pattern Recogn. 2018;77:354–77.

    Article  ADS  Google Scholar 

  80. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition; 2014. ar**v preprint ar**v:1409.1556.

  81. Dionisio-Ortega S, Rojas-Perez LO, Martinez-Carranza J, Cruz-Vega I. A deep learning approach towards autonomous flight in forest environments. In: 2018 International Conference on Electronics, Communications and Computers (CONIELECOMP). IEEE; 2018. p. 139–44.

  82. Espejo-Garcia B, Mylonas N, Athanasakos L, Fountas S. Improving weeds identification with a repository of agricultural pre-trained deep neural networks. Comput Electron Agric. 2020;175:105593.

    Article  Google Scholar 

  83. Siddiqui SA, Salman A, Malik MI, Shafait F, Mian A, Shortis MR, Harvey ES. Automatic fish species classification in underwater videos: exploiting pre-trained deep neural network models to compensate for limited labelled data. ICES J Mar Sci. 2018;75(1):374–89.

    Article  Google Scholar 

  84. Roshanzamir A, Aghajan H, Soleymani Baghshah M. Transformer-based deep neural network language models for Alzheimer’s disease risk assessment from targeted speech. BMC Med Inform Decis Mak. 2021;21(1):1–14.

    Article  Google Scholar 

  85. Lee H, Ho H, Zhou Y. Deep learning-based monocular obstacle avoidance for unmanned aerial vehicle navigation in tree plantations. J Intell Robot Syst. 2021;101(1):1–18.

    Article  Google Scholar 

  86. Vanegas F, Gaston KJ, Roberts J, Gonzalez F. A framework for UAV navigation and exploration in GPS-denied environments. In: 2019 IEEE Aerospace Conference. IEEE; 2019. p. 1–6.

  87. Jung S, Hwang S, Shin H, Shim DH. Perception, guidance, and navigation for indoor autonomous drone racing using deep learning. IEEE Robot Autom Lett. 2018;3(3):2539–44.

    Article  Google Scholar 

  88. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M. ImageNet large scale visual recognition challenge. Int J Comput Vis. 2015;115(3):211–52.

    Article  MathSciNet  Google Scholar 

  89. Padhy RP, Verma S, Ahmad S, Choudhury SK, Sa PK. Deep neural network for autonomous UAV navigation in indoor corridor environments. Procedia Comput Sci. 2018;133:643–50.

    Article  Google Scholar 

  90. Jenssen R, Roverso D. Automatic autonomous vision-based power line inspection: a review of current status and the potential role of deep learning. Int J Electr Power Energy Syst. 2018;99:107–20.

    Article  Google Scholar 

  91. Dai Z, Yi J, Zhang Y, Zhou B, He L. Fast and accurate cable detection using CNN. Appl Intell. 2020;50(12):4688–707.

    Article  Google Scholar 

  92. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016. p. 770–8.

  93. Giusti A, Guzzi J, Cireşan DC, He F-L, Rodríguez JP, Fontana F, Faessler M, Forster C, Schmidhuber J, Di Caro G. A machine learning approach to visual perception of forest trails for mobile robots. IEEE Robot Autom Lett. 2015;1(2):661–7.

    Article  Google Scholar 

  94. Zhang X, Zhang L, Pei H, Lewis FL. Part-based multi-task deep network for autonomous indoor drone navigation. Trans Inst Meas Control. 2020;42(16):3243–53.

    Article  Google Scholar 

  95. Kupervasser O, Kutomanov H, Levi O, Pukshansky V, Yavich R. Using deep learning for visual navigation of drone with respect to 3D ground objects. Mathematics. 2020;8(12):2140.

    Article  Google Scholar 

  96. Liu Y, **. Graph Models. 2021;116:101108.

    Article  Google Scholar 

  97. Opromolla R, Inchingolo G, Fasano G. Airborne visual detection and tracking of cooperative UAVs exploiting deep learning. Sensors. 2019;19(19):4332.

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  98. Waqas A, Kang D, Cha Y-J. Deep learning-based obstacle-avoiding autonomous UAVs with fiducial marker-based localization for structural health monitoring. Struct Health Monit. 2023;14759217231177314.

  99. Chakravarty P, Kelchtermans K, Roussel T, Wellens S, Tuytelaars T, Van Eycken L. CNN-based single image obstacle avoidance on a quadrotor. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE; 2017. p. 6369–74.

  100. Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA. Deep reinforcement learning: a brief survey. IEEE Signal Process Mag. 2017;34(6):26–38.

    Article  ADS  Google Scholar 

  101. Sutton RS, Barto AG. Reinforcement learning: an introduction. Cambridge: MIT Press; 2018.

    Google Scholar 

  102. Imanberdiyev N, Fu C, Kayacan E, Chen I-M. Autonomous navigation of UAV by using real-time model-based reinforcement learning. In: 2016 14th International Conference on Control, Automation, Robotics and Vision (ICARCV). IEEE; 2016. p. 1–6.

  103. Pham HX, La HM, Feil-Seifer D, Van Nguyen L. Reinforcement learning for autonomous UAV navigation using function approximation. In: 2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR). IEEE; 2018. p. 1–6.

  104. Ma Z, Wang C, Niu Y, Wang X, Shen L. A saliency-based reinforcement learning approach for a UAV to avoid flying obstacles. Robot Auton Syst. 2018;100:108–18.

    Article  Google Scholar 

  105. Maciel-Pearson BG, Marchegiani L, Akcay S, Atapour-Abarghouei A, Garforth J, Breckon TP. Online deep reinforcement learning for autonomous UAV navigation and exploration of outdoor environments; 2019. ar**v preprint ar**v:1912.05684.

  106. Koch W, Mancuso R, West R, Bestavros A. Reinforcement learning for UAV attitude control. ACM Trans Cybern-Phys Syst. 2019;3(2):1–21.

    Article  Google Scholar 

  107. Wang C, Wang J, Wang J, Zhang X. Deep-reinforcement-learning-based autonomous UAV navigation with sparse rewards. IEEE Internet Things J. 2020;7(7):6180–90.

    Article  Google Scholar 

  108. Zhou S, Li B, Ding C, Lu L, Ding C. An efficient deep reinforcement learning framework for UAVs. In: 2020 21st International Symposium on Quality Electronic Design (ISQED). IEEE; 2020. p. 323–8.

  109. Bouhamed O, Ghazzai H, Besbes H, Massoud Y. Autonomous UAV navigation: a DDPG-based deep reinforcement learning approach. In: 2020 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE; 2020. p. 1–5.

  110. Agarwal V, Tewari RR. Improving energy efficiency in UAV attitude control using deep reinforcement learning. J Sci Res. 2021;65(3):209–19.

    Google Scholar 

  111. Tong G, Jiang N, Biyue L, ** Z, Ya W, Wenbo D. UAV navigation in high dynamic environments: a deep reinforcement learning approach. Chin J Aeronaut. 2021;34(2):479–89.

    Article  Google Scholar 

  112. Fu C, Xu X, Zhang Y, Lyu Y, **a Y, Zhou Z, Wu W. Memory-enhanced deep reinforcement learning for UAV navigation in 3D environment. Neural Comput Appl. 2022;34(17):14599–607.

    Article  Google Scholar 

  113. Sun Q, Fang J, Zheng WX, Tang Y. Aggressive quadrotor flight using curiosity-driven reinforcement learning. IEEE Trans Ind Electron. 2022;69(12):13838–48.

    Article  Google Scholar 

  114. Walker O, Vanegas F, Gonzalez F, Koenig S. A deep reinforcement learning framework for UAV navigation in indoor environments. In: 2019 IEEE Aerospace Conference. IEEE; 2019. p. 1–14.

Download references

Acknowledgements

The authors would like to thank the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), an agency of the Brazilian Ministry of Science, Technology, Innovations and Communications that supports scientific and technological development, as well as the Fundação de Amparo à Pesquisa e Inovação de Minas Gerais (FAPEMIG), an agency of the State of MG, Brazil, that supports scientific and technological development-for financing this project. Mr. Fagundes would like to thank the Fundação Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), an agency of the Brazilian Ministry of Education that supports human resources perfection, in where this work is inserted, and MSc. de Carvalho would also like to thank the Fundação de Amparo à Pesquisa de Minas Gerais (FAPEMIG) for the scholarship that allowed him to develop his MSc. and PhD. studies, respectively.

Funding

This study was funded by Fundação de Amparo à Pesquisa e Inovação de Minas Gerais (FAPEMIG) (Grant number APQ-02573-21 edital \(\hbox {N}^\circ\)001/2021 - Demanda Universal) and by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) (Grant number 401816/2021-4 Chamada CNPq/MCTI/SEMPI \(\hbox {N}^\circ\)14/2021).

Author information

Authors and Affiliations

  1. Núcleo de Especialização em Robótica, Department of Electrical Engineering, Graduate Program in Computer Science, Federal University of Viçosa-UFV, Av. Petter Henry Rolfs, Viçosa, MG, 36570-900, Brazil

    Leonardo A. Fagundes-Junior, Kevin B. de Carvalho, Ricardo S. Ferreira & Alexandre S. Brandão

Authors
  1. Leonardo A. Fagundes-Junior
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kevin B. de Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ricardo S. Ferreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alexandre S. Brandão
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leonardo A. Fagundes-Junior.

Ethics declarations

Conflict of interest

Leonardo A. Fagundes-Junior declares that he has no conflict of interest. Kevin B. de Carvalho declares that he has no conflict of interest. Ricardo S. Ferreira declares that he has no conflict of interest. Alexandre S. Brandão declares that he has no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fagundes-Junior, L.A., de Carvalho, K.B., Ferreira, R.S. et al. Machine Learning for Unmanned Aerial Vehicles Navigation: An Overview. SN COMPUT. SCI. 5, 256 (2024). https://doi.org/10.1007/s42979-023-02592-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-023-02592-5

Keywords

Search

Navigation