Abstract
The recognition and detection of unmanned aerial vehicles (UAV) usually face the difficulty of insufficient samples. Given a limited number of real UAV images, it is a challenging task to generate virtual UAV images to enrich both the diversity and quantity of training samples. Aiming at this problem, we propose a novel attribute-matched generative adversarial network (AMGAN) that can migrate a UAV object from a single background to a complex background. AMGAN consists of three parts: the basic network, the pairing network, and the background constraint. Image attributes are first disentangled and then reorganized by the basic network, which is prone to attribute collapse. Then the pairing network introduces attribute-level discriminators to make the same-type attributes match each other correctly. Furthermore, the background constraint is added to guide model convergence and eliminate the attribute residue problem. Qualitative experimental results show that AMGAN can generate a large number of high-fidelity virtual UAV images in various backgrounds. Quantitative experimental results on small-scale datasets demonstrate that when these generated images are used for data augmentation, both the diversity and quantity of samples can be greatly increased, boosting the UAV recognition performance.
Similar content being viewed by others
References
Zhao J, Zhang J, Li D, Wang D (2022) Vision-based anti-UAV detection and tracking. IEEE Trans Intell Transport Sys 23(12):25323–25334
Jiang N, Wang K, Peng X, Yu X, Wang Q, **ng J, Li G, Ye Q, Jiao J, Han Z et al (2021) Anti-UAV: a large-scale benchmark for vision-based UAV tracking. IEEE Trans Multim. https://doi.org/10.1109/TMM.2021.3128047
Fu C, Li B, Ding F, Lin F, Lu G (2021) Correlation filters for unmanned aerial vehicle-based aerial tracking: a review and experimental evaluation. IEEE Geosci Remote Sens Magaz 10(1):125–160
Sheu B-H, Chiu C-C, Lu W-T, Huang C-I, Chen W-P (2019) Development of UAV tracing and coordinate detection method using a dual-axis rotary platform for an anti-UAV system. Appl Sci 9(13):2583
Hu Y, Wu X, Zheng G, Liu X (2019) Object detection of UAV for anti-UAV based on improved YOLO v3. In: 2019 Chinese control conference (CCC), IEEE, pp. 8386–8390
Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90
Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1–48
Takahashi R, Matsubara T, Uehara K (2019) Data augmentation using random image crop** and patching for deep CNNs. IEEE Trans Circ Sys Video Technol 30(9):2917–2931
Zhang H, Cisse M, Dauphin YN, Lopez-Paz D (2017) mixup: beyond empirical risk minimization. ar**v preprint ar**v:1710.09412
Verma V, Lamb A, Beckham C, Najafi A, Mitliagkas I, Lopez-Paz D, Bengio Y (2019) Manifold mixup: better representations by interpolating hidden states. In: International conference on machine learning, pp. 6438–6447. PMLR
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial networks. Commun the ACM 63(11):139–144
Mirza M, Osindero S (2014) Conditional generative adversarial nets. ar**v preprint ar**v:1411.1784
Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. ar**v preprint ar**v:1511.06434
Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: International conference on machine learning, IEEE, pp. 214–223
Antoniou A, Storkey A, Edwards H (2017) Data augmentation generative adversarial networks. ar**v preprint ar**v:1711.04340
**ong W, He Y, Zhang Y, Luo W, Ma L, Luo J (2020) Fine-grained image-to-image transformation towards visual recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5840–5849
Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp. 2223–2232
Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134
Choi Y, Choi M, Kim M, Ha J-W, Kim S, Choo J (2018) Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8789–8797
Chen X, Xu C, Yang X, Song L, Tao D (2018) Gated-gan: adversarial gated networks for multi-collection style transfer. IEEE Trans Image Process 28(2):546–560
Zhou S, **ao T, Yang Y, Feng D, He Q, He W (2017) Genegan: learning object transfiguration and attribute subspace from unpaired data. ar**v preprint ar**v:1705.04932
Zhao S, Liu Z, Lin J, Zhu J-Y, Han S (2020) Differentiable augmentation for data-efficient GAN training. Adv Neur Inf Process Sys 33:7559–7570
Li Y, Singh KK, Ojha U, Lee YJ (2020) Mixnmatch: multifactor disentanglement and encoding for conditional image generation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8039–8048
Jha AH, Anand S, Singh M, Veeravasarapu VR (2018) Disentangling factors of variation with cycle-consistent variational auto-encoders. In: Proceedings of the European conference on computer vision (ECCV), pp. 805–820
Liu M-Y, Breuel T, Kautz J (2017) Unsupervised image-to-image translation networks. Advances in neural information processing systems, 30
Zhang H, Goodfellow I, Metaxas D, Odena A (2019) Self-attention generative adversarial networks. In: International conference on machine learning, PMLR, pp. 7354–7363
Mo S, Cho M, Shin J (2020) Freeze the discriminator: a simple baseline for fine-tuning gans. ar**v preprint ar**v:2002.10964
Wen J, Shen Y, Yang J (2022) Multi-view gait recognition based on generative adversarial network. Neural Process Lett 54(3):1855–1877
Huang X, Liu M-Y, Belongie S, Kautz J (2018) Multimodal unsupervised image-to-image translation. In: Proceedings of the European conference on computer vision (ECCV), pp. 172–189
Lee H-Y, Tseng H-Y, Mao Q, Huang J-B, Lu Y-D, Singh M, Yang M-H (2020) Drit++: diverse image-to-image translation via disentangled representations. Int J Comput Vision 128(10):2402–2417
Elasri M, Elharrouss O, Al-Maadeed S, Tairi H (2022) Image generation: a review. Neural Process Lett 54(5):4609–4646
Wah C, Branson S, Welinder P, Perona P, Belongie S (2011) The Caltech-UCSD Birds-200-2011 dataset
Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, pp. 248–255
**a G-S, Bai X, Ding J, Zhu Z, Belongie S, Luo J, Datcu M, Pelillo M, Zhang L (2018) Dota: a large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3974–3983
Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI conference on artificial intelligence
Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30
Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X (2016) Improved techniques for training gans. Advances in neural information processing systems 29
Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: European conference on computer vision. Springer, pp. 740–755
Li J, Murray J, Ismaili D, Schindler K, Albl C (2020) Reconstruction of 3D flight trajectories from ad-hoc camera networks. In: 2020 IEEE/RSJ International conference on intelligent robots and systems (IROS), IEEE, pp. 1621–1628
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant No. 61201238, in part by the Aeronautical Science Foundation of China under Grant No. 201801P6002, and in part by the Fundamental Research Funds for the Central Universities under Grant No. 3072022CF0802.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yang, Z., Jia, X., Shen, Y. et al. AMGAN: An Attribute-Matched Generative Adversarial Network for UAV Virtual Sample Generation. Neural Process Lett 55, 8131–8149 (2023). https://doi.org/10.1007/s11063-023-11304-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-023-11304-2