SpringerLink
Log in

An outstanding adaptive multi-feature fusion YOLOv3 algorithm for the small target detection in remote sensing images

  • Short Paper
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Deep learning-based target detection for optical remote sensing images is a significant research direction in the field of image processing. Different from natural images, remote sensing images are characterized by complex backgrounds, similarity of characteristics between various classes and diverse target scales. In this paper, we propose an adaptive multi-feature fusion YOLOv3 remote sensing small target detection algorithm to cope with these features. In the proposed algorithm, the shallow semantic information is extracted by the accessory feature extraction network, and fused with the deep features extracted by Darknet-53 down-sampling to enrich the semantic and spatial information of the feature layers on the auxiliary network. In addition, the shallow features are filtered using the adaptive feature selection module to refine the effective feature information. A cross-layer feature fusion module is proposed to fuse different feature layers to enhance the connection between the semantic information of feature contexts to obtain more information about the characteristics of small targets. To test the effectiveness of the proposed algorithm, it is validated on the Pascal voc2007 dataset. The experimental results show that the detection accuracy of the proposed algorithm could achieve 88.3%, and evidently superior to the original YOLOv3 algorithm. Finally, the proposed algorithm is applied to detect the small target in remote sensing images. The detection results show that compared with the original YOLOv3 algorithm, the mean average precision(mAP) of the proposed algorithm is improved by 2.6%, which can effectively detect more small targets and significantly improve the detection accuracy of small targets than other classical algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Availability of data and material

This paper shows the experimental results of the proposed algorithm. Code availability: Not applicable.

References

  1. Tondewad MPS, Dale MMP (2020) Remote sensing image registration methodology: review and discussion. Proc Comput Sci 171:2390–2399. https://doi.org/10.1016/j.procs.2020.04.259 (third International Conference on Computing and Network Communications (CoCoNet’19))

    Article  Google Scholar 

  2. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE conference on computer vision and pattern recognition vol 2014, pp 580–587. https://doi.org/10.1109/CVPR.2014.81

  3. Śmieja M, Tabor J, Spurek P (2019) Svm with a neutral class. Pattern Anal Appl. https://doi.org/10.1007/s10044-017-0654-3

    Article  Google Scholar 

  4. Girshick R (2015) Fast r-cnn. In: IEEE international conference on computer vision (ICCV), vol 2015, pp 1440–1448. https://doi.org/10.1109/ICCV.2015.169

  5. Ren S, He K, Girshick R, Sun J (2017) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  6. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: IEEE conference on computer vision and pattern recognition (CVPR) vol 2016, pp 779–788. https://doi.org/10.1109/CVPR.2016.91

  7. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision—ECCV 2016. Springer International Publishing, Cham, pp 21–37

    Chapter  Google Scholar 

  8. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition (CVPR) vol 2016, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  9. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651. https://doi.org/10.1109/TPAMI.2016.2572683

    Article  Google Scholar 

  10. Fu C-Y, Liu W, Ranga A, Tyagi A, Berg AC (2017) Dssd : deconvolutional single shot detector. ar**v:1701.06659

  11. Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: IEEE conference on computer vision and pattern recognition (CVPR) vol 2017, pp 6517–6525. https://doi.org/10.1109/CVPR.2017.690

  12. **anbao C, Guihua Q, Yu J, Zhaomin Z (2021) An improved small object detection method based on yolo v3. Pattern Anal Appl. https://doi.org/10.1007/s10044-021-00989-7

    Article  Google Scholar 

  13. Seferbekov S, Iglovikov V, Buslaev A, Shvets A (2018) Feature pyramid network for multi-class land segmentation. In: IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW) vol 2018, pp 272–2723. https://doi.org/10.1109/CVPRW.2018.00051

  14. Pang J, Li C, Shi J, Xu Z, Feng H (2019) ¡inline-formula¿ ¡tex-math notation=“latex’’¿\(\cal{R}2\) ¡/tex-math¿¡/inline-formula¿-cnn: Fast tiny object detection in large-scale remote sensing images. IEEE Trans Geosci Remote Sens 57(8):5512–5524. https://doi.org/10.1109/TGRS.2019.2899955

    Article  Google Scholar 

  15. Yang X, Yang J, Yan J, Zhang Y, Zhang T, Guo Z, Sun X, Fu K (2019) Scrdet: Towards more robust detection for small, cluttered and rotated objects. In: IEEE/CVF international conference on computer vision (ICCV) vol 2019, pp 8231–8240. https://doi.org/10.1109/ICCV.2019.00832

  16. Jiang W, Zhang C, Zhang S, Liu W, University LT, School G, University LT (2019) Multiscale feature map fusion algorithm for target detection. J Image Graph. https://doi.org/10.11834/jig.190021

    Article  Google Scholar 

  17. Wang P, Sun X, Diao W, Fu K (2020) Fmssd: feature-merged single-shot detection for multiscale objects in large-scale remote sensing imagery. IEEE Trans Geosci Remote Sens 58(5):3377–3390. https://doi.org/10.1109/TGRS.2019.2954328

    Article  Google Scholar 

  18. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2015) Object detectors emerge in deep scene cnns. ar**v:1412.6856

  19. Herout A, Hradiš M, Zemčík P (2012) Enms: early non-maxima suppression. Pattern Anal Appl. https://doi.org/10.1007/s10044-011-0213-2

    Article  Google Scholar 

  20. Li X, Wang W, Hu X, Yang J (2019) Selective kernel networks. In: IEEE/CVF conference on computer vision and pattern recognition (CVPR) vol 2019, pp 510–519. https://doi.org/10.1109/CVPR.2019.00060

  21. Li K, Wan G, Cheng G, Meng L, Han J (2020) Object detection in optical remote sensing images: a survey and a new benchmark. ISPRS J Photogramm Remote Sens 159:296–307. https://doi.org/10.1016/j.isprsjprs.2019.11.023

    Article  Google Scholar 

  22. Law H, Deng J (2019) Cornernet: detecting objects as paired keypoints. ar**v:1808.01244

  23. Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2020) Focal loss for dense object detection. IEEE Trans Pattern Anal Mach Intell 42(2):318–327. https://doi.org/10.1109/TPAMI.2018.2858826

    Article  Google Scholar 

  24. Cheng G, Si Y, Hong H, Yao X, Guo L (2021) Cross-scale feature fusion for object detection in optical remote sensing images. IEEE Geosci Remote Sens Lett 18(3):431–435. https://doi.org/10.1109/LGRS.2020.2975541

    Article  Google Scholar 

Download references

Acknowledgements

This work was under the funding of the National Natural Science Foundation of China (61973264) and the Natural Science Foundation of Hebei Province (F2020203003).

Funding

This work was under the funding of the National Natural Science Foundation of China (61973264) and the Natural Science Foundation of Hebei Province (F2020203003).

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Yanshan University, Qinhuangdao, 066004, China

    Guoqiang Li, **nyu Hao, Linlin Zha & Anbang Chen

  2. Key Lab of Industrial Computer Control Engineering of Hebei Province, Yanshan University, Qinhuangdao, 066004, China

    **nyu Hao, Linlin Zha & Anbang Chen

Authors
  1. Guoqiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. **nyu Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Linlin Zha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anbang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

(1) We propose an improved YOLOv3 algorithm based on adaptive multi-feature fusion adaptive multi-feature fusion. (2) An accessory feature extraction network is introduced into the side branch of the DarkNet-53 backbone network. (3) An adaptive feature selection module, which assigns weights to feature maps with different sensory field sizes and extracts discriminative features, is proposed. (4) A cross-layer feature fusion module is introduced into the feature pyramid network to enhance the connection between feature contextual information. (5) Experimental results illustrate the effectiveness of the proposed method. The proposed method can detect more small targets and obtain higher confidence scores than original yolov3 algorithm.

Corresponding author

Correspondence to **nyu Hao.

Ethics declarations

Conflicts of interest

The authors declared that they have no conflicts of interest in this work. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, G., Hao, X., Zha, L. et al. An outstanding adaptive multi-feature fusion YOLOv3 algorithm for the small target detection in remote sensing images. Pattern Anal Applic 25, 951–962 (2022). https://doi.org/10.1007/s10044-022-01072-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-022-01072-5

Keywords

Mathematics Subject Classification

Search

Navigation