Abstract
Unmanned Aerial Vehicles (UAVs) possess significant advantages in terms of mobility and range compared to traditional surveillance cameras. Human action detection from UAV images has the potential to assist in various fields, including search and rescue operations. However, UAV images present challenges such as varying heights, angles, and the presence of small objects. Additionally, they can be affected by adverse illumination and weather conditions. In this paper, we propose a Multi-level Attention network with Weather Suppression for all-weather action detection in UAV rescue scenarios. The Weather Suppression module effectively mitigates the impact of illumination and weather, while the Multi-level Attention module enhances the model’s performance in detecting small objects. We conducted detection experiments under both normal and synthetic harsh conditions, and the results demonstrate that our model achieves state-of-the-art performance. Furthermore, a comparison of relevant metrics reveals that our model strikes a balance between size and complexity, making it suitable for deployment on UAV platforms. The conducted ablation experiments also highlight the significant contribution of our proposed modules.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Barekatain, M., Martí, M., et al: Okutama-action: an aerial view video dataset for concurrent human action detection. In: CVPR Workshops, pp. 2153–2160 (2017)
Bozcan, I., Kayacan, E.: AU-AIR: a multi-modal unmanned aerial vehicle dataset for low altitude traffic surveillance. In: ICRA, pp. 8504–8510 (2020)
Bozcan, I., Kayacan, E.: Context-dependent anomaly detection for low altitude traffic surveillance. CoRR abs/2104.06781 (2021)
Budiharto, W., Gunawan, A.A.S., et al.: Fast object detection for quadcopter drone using deep learning. In: ICCCS, pp. 192–195 (2018)
Cai, Y., Du, D., et al.: Guided attention network for object detection and counting on drones. In: MM, pp. 709–717 (2020)
Cai, Z., Vasconcelos, N.: Cascade R-CNN: delving into high quality object detection. In: CVPR, pp. 6154–6162 (2018)
Du, D., Qi, Y., Yu, H., Yang, Y., Duan, K., Li, G., Zhang, W., Huang, Q., Tian, Q.: The unmanned aerial vehicle benchmark: object detection and tracking. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 375–391. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_23
Erdelj, M., Natalizio, E.: UAV-assisted disaster management: applications and open issues. In: ICNC, pp. 1–5 (2016)
Everingham, M., Gool, L.V., et al.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88, 303–338 (2010)
Ge, Z., Liu, S., et al.: YOLOX: exceeding YOLO series in 2021. CoRR abs/2107.08430 (2021)
Ghiasi, G., Lin, T., Le, Q.V.: NAS-FPN: learning scalable feature pyramid architecture for object detection. In: CVPR, pp. 7036–7045 (2019)
Girshick, R.B.: Fast R-CNN. In: ICCV, pp. 1440–1448 (2015)
Girshick, R.B., Donahue, J., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR, pp. 580–587 (2014)
Goodfellow, I.J., Pouget-Abadie, J., et al.: Generative adversarial networks. Commun. ACM. 63, 139–144 (2020)
He, K., Gkioxari, G., et al.: Mask R-CNN. In: ICCV, pp. 2980–2988 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)
Land, E.H.: The retinex theory of color vision. Sci. Am. 237, 108–129 (1977)
Li, T., Liu, J., et al.: UAV-human: a large benchmark for human behavior understanding with unmanned aerial vehicles. In: CVPR, pp. 16266–16275 (2021)
Li, Z., Liu, X., et al.: A lightweight multi-scale aggregated model for detecting aerial images captured by UAVs. J. Vis. Commun. Image Represent. 77, 103058 (2021)
Liang, X., Zhang, J., et al.: Small object detection in unmanned aerial vehicle images using feature fusion and scaling-based single shot detector with spatial context analysis. IEEE Trans. Circuits Syst. Video Technol. 30, 1758–1770 (2020)
Lin, T., Goyal, P., et al.: Focal loss for dense object detection. In: ICCV, pp. 2999–3007 (2017)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Liu, C., Szirányi, T.: Real-time human detection and gesture recognition for on-board UAV rescue. Sensors. 21, 2180 (2021)
Liu, S., Huang, D., Wang, Y.: Learning spatial fusion for single-shot object detection. CoRR abs/1911.09516 (2019)
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Mabrouk, A.B., Zagrouba, E.: Abnormal behavior recognition for intelligent video surveillance systems: a review. Expert Syst. Appl. 91, 480–491 (2018)
Mishra, B., Garg, D., et al.: Drone-surveillance for search and rescue in natural disaster. Comput. Commun. 156, 1–10 (2020)
Moranduzzo, T., Melgani, F.: Detecting cars in UAV images with a catalog-based approach. IEEE Trans. Geosci. Remote. Sens. 52, 6356–6367 (2014)
Du, D., et al.: The unmanned aerial vehicle benchmark: object detection and tracking. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 375–391. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_23
Papaioannidis, C., Mademlis, I., Pitas, I.: Autonomous UAV safety by visual human crowd detection using multi-task deep neural networks. In: ICRA, pp. 11074–11080 (2021)
Perera, A.G., Law, Y.W., Chahl, J.: Drone-action: an outdoor recorded drone video dataset for action recognition. Drones. 3, 82 (2019)
Perera, A.G., Law, Y.W., Chahl, J.: UAV-GESTURE: a dataset for UAV control and gesture recognition. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11130, pp. 117–128. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11012-3_9
Perlin, K.: Improving noise. In: ACM Transactions of Graph, pp. 681–682, July 2002
Radovic, M., Adarkwa, O., Wang, Q.: Object recognition in aerial images using convolutional neural networks. J. Imaging. 3, 21 (2017)
Redmon, J., Divvala, S.K., et al.: You only look once: unified, real-time object detection. In: CVPR, pp. 779–788 (2016)
Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. CoRR abs/1804.02767 (2018)
Ren, S., He, K., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. In: NIPS, pp. 91–99 (2015)
Rohan, A., Rabah, M., Kim, S.H.: Convolutional neural network-based real-time object detection and tracking for parrot AR drone 2, pp. 69575–69584 . IEEE Access (2019)
Semsch, E., Jakob, M., et al.: Autonomous UAV surveillance in complex urban environments. In: IAT, pp. 82–85 (2009)
Sevo, I., Avramovic, A.: Convolutional neural network based automatic object detection on aerial images. IEEE Geosci. Remote. Sens. Lett. 13, 740–744 (2016)
Sommer, L.W., Schuchert, T., Beyerer, J.: Fast deep vehicle detection in aerial images. In: WACV, pp. 311–319 (2017)
Szeliski, R.: Computer Vision: Algorithms and Applications, 1st edn. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-030-34372-9
Tan, M., Pang, R., Le, Q.V.: Efficientdet: scalable and efficient object detection. In: CVPR, pp. 10778–10787 (2020)
Tijtgat, N., Volckaert, B., Turck, F.D.: Real-time hazard symbol detection and localization using UAV imagery. In: VTC, pp. 1–5 (2017)
Wen, X., Shao, L., et al.: Efficient feature selection and classification for vehicle detection. IEEE Trans. Circuits Syst. Video Technol. 25, 508–517 (2015)
Wu, Z., Suresh, K., et al.: Delving into robust object detection from unmanned aerial vehicles: a deep nuisance disentanglement approach. In: ICCV, pp. 1201–1210 (2019)
**e, H., Chen, Y., Shin, H.: Context-aware pedestrian detection especially for small-sized instances with deconvolution integrated faster RCNN (DIF R-CNN). Appl. Intell. 49, 1200–1211 (2019)
**e, H., Shin, H.: Two-stream small-scale pedestrian detection network with feature aggregation for drone-view videos. Multidimens. Syst. Signal Process. 32, 897–913 (2021)
Ye, J., Fu, C., et al.: Darklighter: light up the darkness for UAV tracking. In: IROS, pp. 3079–3085 (2021)
Yu, W., Yang, T., Chen, C.: Towards resolving the challenge of long-tail distribution in UAV images for object detection. In: WACV, pp. 3257–3266 (2021)
Zhang, C., Ge, S., et al.: Accurate UAV tracking with distance-injected overlap maximization. In: MM, pp. 565–573 (2020)
Zhang, X., Izquierdo, E., Chandramouli, K.: Dense and small object detection in UAV vision based on cascade network. In: ICCV, pp. 118–126 (2019)
Zhu, P., Wen, L., et al: Vision meets drones: a challenge. CoRR abs/1804.07437 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, Y., Li, B., Sammut, C., Yao, L. (2024). Multi-level Attention Network with Weather Suppression for All-Weather Action Detection in UAV Rescue Scenarios. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1963. Springer, Singapore. https://doi.org/10.1007/978-981-99-8138-0_43
Download citation
DOI: https://doi.org/10.1007/978-981-99-8138-0_43
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8137-3
Online ISBN: 978-981-99-8138-0
eBook Packages: Computer ScienceComputer Science (R0)