Abstract
Multi-label classification is a generalization of single-label classification, where an unseen sample is automatically assigned a subset of semantically relevant labels from a given vocabulary. In parallel, recent research has demonstrated the impact of adversarial examples, which are modifications of original samples and aim at fooling machine learning models. Unlike existing adversary generation techniques which are specific to single-label data and mostly assume the availability of training data and/or model to the attacker, in this paper, we propose a generalized adversary generation mechanism by generating worst-case perturbation. This perturbation, when added to the feature vector of the original sample, generates an adversarial sample without the need for the availability of either training data or model to the attacker. Next, for the first time as per our knowledge, we study and demonstrate the effect of feature normalization as a defense mechanism against adversarial attacks. Extensive experiments show the effectiveness of our adversarial attack and defense mechanisms using state-of-the-art max-margin multi-label classification algorithms on two benchmark datasets.
RKG contributed to this work while he was a student at IIT Jodhpur.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Babbar R, Schölkopf B (2017) Dismec: distributed sparse machines for extreme multi-label classification. In: Proceedings of the tenth ACM international conference on web search and data mining. WSDM ’17, Association for Computing Machinery, New York, NY, USA, pp. 721–729 (2017). https://doi.org/10.1145/3018661.3018741
Babbar R, Schölkopf B (2019) Data scarcity, robustness and extreme multi-label classification. Mach Learn 108(8):1329–1351 (2019). https://doi.org/10.1007/s10994-019-05791-5
Bhatia K, Dahiya K, Jain H, Kar P, Mittal A, Prabhu Y, Varma M (2016) The extreme classification repository: multi-label datasets and code. http://manikvarma.org/downloads/XC/XMLRepository.html
Chen SF, Chen YC, Yeh CK, Wang YC (2018) Order-free RNN with visual attention for multi-label classification (2018). https://aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16114/16253
Chen ZM, Wei XS, Wang P, Guo Y (2019) Multi-label image recognition with graph convolutional networks. In: CVPR, pp 5177–5186
Dutta A, Verma Y, Jawahar CV (2020) Recurrent image annotation with explicit inter-label dependencies. In: ECCV, pp 191–207
Duygulu P, Barnard K, de Freitas JFG, Forsyth DA (2002) Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In: Heyden A, Sparr G, Nielsen M, Johansen P (eds) Computer vision—ECCV 2002. Springer, Berlin, Heidelberg, pp 97–112
Fellbaum C (1998) WordNet: an electronic lexical database (1998). https://wordnet.princeton.edu/
Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. CoRR ar**v:1412.6572
Hariharan B, Zelnik-Manor L, Vishwanathan SVN, Varma M (2010) Large scale max-margin multi-label classification with priors. In: Proceedings of the international conference on machine learning, June 2010
ImageClef: ImageClef—IAPR TC-12 benchmark. https://www.imageclef.org/photodata
Jeon J, Lavrenko V, Manmatha R (2003) Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th annual international acm sigir conference on research and development in informaion retrieval. SIGIR ’03, Association for Computing Machinery, New York, NY, USA, pp 119–126. https://doi.org/10.1145/860435.860459
Lavrenko V, Feng SL, Manmatha R (2004) Multiple bernoulli relevance models for image and video annotation. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, vol 2, July 2004. IEEE Computer Society, Los Alamitos, CA, USA, pp 1002–1009. https://doi.org/10.1109/CVPR.2004.171, https://doi.ieeecomputersociety.org/10.1109/CVPR.2004.171
Lavrenko V, Manmatha R, Jeon J (2004) A model for learning the semantics of pictures. In: Thrun S, Saul LK, Schölkopf B (eds) Advances in neural information processing systems, vol 16. MIT Press, pp 553–560. http://papers.nips.cc/paper/2474-a-model-for-learning-the-semantics-of-pictures.pdf
Liu X, Cheng M, Zhang H, Hsieh CJ (2018) Towards robust neural networks via random self-ensemble. Ar**v ar**v:1712.00673
Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2018) Towards deep learning models resistant to adversarial attacks. Ar**v ar**v:1706.06083
Makadia A, Pavlovic V, Kumar S (2008) A new baseline for image annotation. In: Forsyth D, Torr P, Zisserman A (eds) Computer vision—ECCV 2008. Springer, Berlin, Heidelberg, pp 316–329
Makadia A, Pavlovic V, Kumar S (2010) Baselines for image annotation. Int J Comput Vis 90(1):88–105 (2010). https://doi.org/10.1007/s11263-010-0338-6
Mopuri KR, Ojha U, Garg U, Babu RV (2018) NAG: network for adversary generation. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 742–751
Mori Y, Takahashi H, Oka R (1999) Image-to-word transformation based on dividing and vector quantizing images with words. In: MISRM’99 first international workshop on multimedia intelligent storage and retrieval management. citeseer.ist.psu.edu/368129.html
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: ICLR
Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow IJ, Fergus R (2014) Intriguing properties of neural networks. CoRR ar**v:1312.6199
Tramèr F, Kurakin A, Papernot N, Boneh D, McDaniel P (2018) Ensemble adversarial training: Attacks and defenses. Ar**v ar**v:1705.07204
Verma Y (2019) Diverse image annotation with missing labels. Pattern Recognit. 93:470–484
Verma Y, Jawahar CV (2017) Image annotation by propagating labels from semantic neighbourhoods. Int. J. Comput. Vis. 121(1):126–148
von Ahn L, Dabbish L (2004) Labeling images with a computer game. In: Proceedings of the SIGCHI conference on human factors in computing systems. CHI ’04, Association for Computing Machinery, New York, NY, USA, pp. 319–326 (2004). https://doi.org/10.1145/985692.985733
Wu B, Jia F, Liu W, Ghanem B (2017) Diverse image annotation. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 6194–6202
**e C, Wang J, Zhang Z, Ren Z, Yuille A (2018) Mitigating adversarial effects through randomization. Ar**v ar**v:1711.01991
Zhu F, Li H, Ouyang W, Yu N, Wang X (2017) Learning spatial regularization with image-level supervisions for multi-label image classification. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 2027–2036
Acknowledgements
YV would like to thank the Department of Science and Technology (India) for the INSPIRE Faculty award 2017.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Gupta, R.K., Verma, Y. (2022). Worst-Case Adversarial Perturbation and Effect of Feature Normalization on Max-Margin Multi-label Classifiers. In: Mudenagudi, U., Nigam, A., Sarvadevabhatla, R.K., Choudhary, A. (eds) Proceedings of the Satellite Workshops of ICVGIP 2021. Lecture Notes in Electrical Engineering, vol 924. Springer, Singapore. https://doi.org/10.1007/978-981-19-4136-8_13
Download citation
DOI: https://doi.org/10.1007/978-981-19-4136-8_13
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-4135-1
Online ISBN: 978-981-19-4136-8
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)