SpringerLink
Log in

A hybrid style transfer with whale optimization algorithm model for textual adversarial attack

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Deep learning has been widely used in various research fields. However, researchers have discovered that deep learning models are vulnerable to adversarial attacks. Existing word-level attacks can be seen as a combinatorial optimization problem to effectively conduct textual adversarial attacks, but inappropriate search spaces and search methods may affect attack effectiveness. Sentence-level attacks are successfully used in the field of reading comprehension, but the generated examples sometimes lead to semantic deviation. To address these issues, we propose a hybrid textual adversarial attack method that effectively enhances the performance of textual adversarial attacks. To the best of our knowledge, we are the first to conduct textual adversarial attacks by hybridizing Whale Optimization Algorithm (WOA) with style transfer from multiple sentence and word levels. The WOA is improved by incorporating data characteristics and the Metropolis criterion to escape from local optima and by leveraging the mutation operator to increase population diversity. The improved WOA and style transfer algorithm are fused in a parallel and vertical way. Style transfer can increase population diversity and expand the search space, usually without destroying the semantics and syntax of sentences. The parallel combination improves attack performance by attacking from both word-level and sentence-level perspectives. As a black-box attack model, our method can attack without knowing the internal structure of the model. Compared with the state-of-the-art method, our framework can improve the attack success rate by 6.8%. Additionally, further experiments on grammatical error increase rates, semantic consistency, and transferability demonstrate that our model has excellent performance in many respects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Availability of supporting data

The datasets used or analyzed during the current study are available from the corresponding author on reasonable request.

Notes

  1. https://languagetool.org.

References

  1. Wang Y, Hu X (2022) Machine learning-based image recognition for rural architectural planning and design[J]. Neural Comput Appl, 1–10

  2. Zhang Y, Liu Y, Yang G, Song J (2022) Ssit: a sample selection-based incremental model training method for image recognition. Neural Comput Appl 34(4):3117–3134

    Article  Google Scholar 

  3. Qin P, Zhang C, Dang M (2022) Gvnet: Gaussian model with voxel-based 3d detection network for autonomous driving. Neural Comput Appl 34(9):6637–6645

    Article  Google Scholar 

  4. Rais MS, Zouaidia K, Boudour R (2022) Enhanced decision making in multi-scenarios for autonomous vehicles using alternative bidirectional Q network[J]. Neural Comput Appl 34(18):15981–15996

    Article  Google Scholar 

  5. Szegedy C, Zaremba W, Sutskever I et al. (2013) Intriguing properties of neural networks[J]. Comput Sci. https://doi.org/10.48550/ar**v.1312.6199

    Article  Google Scholar 

  6. Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. Stat 1050:20

    Google Scholar 

  7. Olatunji SO (2019) Improved email spam detection model based on support vector machines. Neural Comput Appl 31(3):691–699

    Article  Google Scholar 

  8. Barushka A, Hajek P (2020) Spam detection on social networks using cost-sensitive feature selection and ensemble-based regularized deep neural networks. Neural Comput Appl 32(9):4239–4257

    Article  Google Scholar 

  9. Yan H, Yi B, Li H, Wu D (2022) Sentiment knowledge-induced neural network for aspect-level sentiment analysis. Neural Comput Appl 34(24):22275–22286

    Article  Google Scholar 

  10. Passalis N, Avramelou L, Seficha S, Tsantekidis A, Doropoulos S, Makris G, Tefas A (2022) Multisource financial sentiment analysis for detecting bitcoin price change indications using deep learning. Neural Comput Appl 34(22):19441–19452

    Article  Google Scholar 

  11. Huang L, Chen W, Liu Y, Zhang H, Qu H (2021) Improving neural machine translation using gated state network and focal adaptive attention network. Neural Comput Appl 33(23):15955–15967

    Article  Google Scholar 

  12. Singh SM, Singh TD (2022) An empirical study of low-resource neural machine translation of manipuri in multilingual settings[J]. Neural Comput Appl 34(17):14823–14844

    Article  Google Scholar 

  13. Hosseini H, Kannan S, Zhang B, Poovendran R (2017) Deceiving Google’s perspective API built for detecting toxic comments. ar**v preprint ar**v:1702.08138

  14. Li L, Ma R, Guo Q, Xue X, Qiu X (2020) Bert-attack: Adversarial attack against Bert using Bert. In: Proceedings of the 2020 conference on empirical methods in natural language processing, pp 6193–6202

  15. Zhang WE, Sheng QZ, Alhazmi A, Li C (2020) Adversarial attacks on deep-learning models in natural language processing: a survey. ACM Trans Intell Syst Technol 11(3):1–41

    Google Scholar 

  16. Wang W, Wang R, Wang L, et al. (2021) Towards a robust deep neural network against adversarial texts: A survey[J]. IEEE Trans Knowledge Data Eng

  17. Belinkov Y, Bisk Y (2018) Synthetic and natural noise both break neural machine translation. In: International conference on learning representations

  18. Ebrahimi J, Rao A, Lowd D, Dou D (2018) Hotflip: white-box adversarial examples for text classification. In: Proceedings of the 56th annual meeting of the association for computational linguistics, pp 31–36

  19. Gil Y, Chai Y, Gorodissky O, Berant J (2019) White-to-black: Efficient distillation of black-box adversarial attacks. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 1373–1379

  20. Alzantot M, Sharma Y, Elgohary A, Ho B-J, Srivastava M, Chang K-W (2018) Generating natural language adversarial examples. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 2890–2896

  21. Ren S, Deng Y, He K, Che W (2019) Generating natural language adversarial examples through probability weighted word saliency. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 1085–1097

  22. ** D, ** Z, Zhou JT, Szolovits P (2020) Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 8018–8025

  23. Tsai Y-T, Yang M-C, Chen H-Y (2019) Adversarial attack on sentiment classification. In: Proceedings of the 2019 ACL workshop BlackboxNLP: analyzing and interpreting neural networks for NLP, pp 233–240

  24. Zang Y, Qi F, Yang C, Liu Z, Zhang M, Liu Q, Sun M (2020) Word-level textual adversarial attacking as combinatorial optimization. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 6066–6080

  25. Yang X, Liu W, Tao D, Liu W (2021) Besa: Bert-based simulated annealing for adversarial text attacks. In: Proceedings of the 30th international joint conference on artificial intelligence, pp. 3293–3299

  26. Jia R, Liang P (2017) Adversarial examples for evaluating reading comprehension systems. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2021–2031

  27. Ribeiro MT, Singh S, Guestrin C (2018) Semantically equivalent adversarial rules for debugging NLP models. In: Proceedings of the 56th annual meeting of the association for computational linguistics, pp 856–865

  28. Wang T, Wang X, Qin Y, Packer B, Li K, Chen J, Beutel A, Chi E (2020) Cat-gen: improving robustness in NLP models via controlled adversarial text generation. In: Proceedings of the 2020 conference on empirical methods in natural language processing

  29. Qi F, Chen Y, Zhang X, Li M, Liu Z, Sun M (2021) Mind the style of text! adversarial and backdoor attacks based on text style transfer. In: Proceedings of the 2021 conference on empirical methods in natural language processing, pp 4569–4580

  30. Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2018) Towards deep learning models resistant to adversarial attacks. In: International conference on learning representations. https://openreview.net/forum?id=rJzIBfZAb

  31. Wu T, Tong L, Vorobeychik Y (2020) Defending against physically realizable attacks on image classification. In: International conference on learning representations. https://openreview.net/forum?id=H1xscnEKDr

  32. Zhou D, Liu T, Han B, Wang N, Peng C, Gao X (2021) Towards defending against adversarial examples via attack-invariant features. In: International conference on machine learning. PMLR, pp 12835–12845

  33. Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67

    Article  Google Scholar 

  34. John V, Mou L, Bahuleyan H, Vechtomova O (2019) Disentangled representation learning for non-parallel text style transfer. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 424–434

  35. Wang K, Hua H, Wan X (2019) Controllable unsupervised text attribute transfer via editing entangled latent representation[J]. Adv Neural Info Process Syst 32

  36. Dai N, Liang J, Qiu X, Huang X-J (2019) Style transformer: unpaired text style transfer without disentangled latent representation. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 5997–6007

  37. He J, Wang X, Neubig G, Berg-Kirkpatrick T (2019) A probabilistic formulation of unsupervised text style transfer. In: International conference on learning representations

  38. Bloomfield L (1926) A set of postulates for the science of language. Language 2(3):153–164

    Article  Google Scholar 

  39. Dong Z, Dong Q (2006) Hownet and the computation of meaning. World Scientific Publishing Co., Inc

  40. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21(6):1087–1092

    Article  ADS  CAS  Google Scholar 

  41. Kirkpatrick S, Gelatt CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680

    Article  ADS  MathSciNet  CAS  PubMed  Google Scholar 

  42. Krishna K, Wieting J, Iyyer M (2020) Reformulating unsupervised style transfer as paraphrase generation. In: Proceedings of the 2020 conference on empirical methods in natural language processing

  43. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I et al (2019) Language models are unsupervised multitask learners. OpenAI Blog 1(8):9

    Google Scholar 

  44. Reimers N, Gurevych I (2019) Sentence-bert: sentence embeddings using siamese bert-networks. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp 3982–3992

  45. Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng AY, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 1631–1642

  46. Maas A, Daly RE, Pham PT, Huang D, Ng AY, Potts C (2011) Learning word vectors for sentiment analysis. In: Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pp 142–150

  47. Bowman SR, Angeli G, Potts C, Manning CD (2015) A large annotated corpus for learning natural language inference. In: Proceedings of the 2015 conference on empirical methods in natural language processing

  48. de Gibert O, Pérez N, García-Pablos A, Cuadros M (2018) Hate speech dataset from a white supremacy forum. In: Proceedings of the 2nd workshop on abusive language online, pp 11–20

  49. Zhang X, Zhao J, LeCun Y (2015) Character-level convolutional networks for text classification[J]. Adv Neural Info Process Syst 28

  50. Conneau A, Kiela D, Schwenk H, Barrault L, Bordes A (2017) Supervised learning of universal sentence representations from natural language inference data. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 670–680

  51. Devlin J, Chang M-W, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 4171–4186

  52. Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2019) Albert: a lite bert for self-supervised learning of language representations. In: International conference on learning representations

  53. Sanh V, Debut L, Chaumond J, Wolf T (2019) Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ar**v preprint ar**v:1910.01108

  54. Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing, pp 1532–1543

  55. Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, et al. (2020) Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pp 38–45

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China, Grant No. 62366060, 61762092, Open Foundation of Yunnan Key Laboratory of Software Engineering under Grant No. 2023SE203, the Major Science and Technology Projects in Yunnan Province, Grant No. 202002AD080047 and 202202AE090019.

Author information

Authors and Affiliations

  1. Yunnan Key Laboratory of Software Engineering, National Pilot School of Software, Yunnan University, Chenggong District, Kunming, 650500, Yunnan, China

    Yan Kang, Jianjun Zhao, Baochen Fan & Wentao **e

  2. National Engineering Laboratory for Surface Transportation Weather Impacts Prevention, Broadvision Engineering Consultants Co., Ltd., Kunming, China

    Xuekun Yang

Authors
  1. Yan Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianjun Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuekun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Baochen Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wentao **e
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuekun Yang.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, Y., Zhao, J., Yang, X. et al. A hybrid style transfer with whale optimization algorithm model for textual adversarial attack. Neural Comput & Applic 36, 4263–4280 (2024). https://doi.org/10.1007/s00521-023-09278-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09278-2

Keywords

Search

Navigation