SpringerLink
Log in

A literature survey on multimodal and multilingual automatic hate speech identification

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Social media is a more common and powerful platform for communication to share views about any topic or article, which consequently leads to unstructured toxic, and hateful conversations. Curbing hate speeches has emerged as a critical challenge globally. In this regard, Social media platforms are using modern statistical tools of AI technologies to process and eliminate toxic data to minimize hate crimes globally. Demanding the dire need, machine and deep learning-based techniques are getting more attention in analyzing these kinds of data. This survey presents a comprehensive analysis of hate speech definitions along with the motivation for detection and standard textual analysis methods that play a crucial role in identifying hate speech. State-of-the-art hate speech identification methods are also discussed, highlighting handcrafted feature-based and deep learning-based algorithms by considering multimodal and multilingual inputs and stating the pros and cons of each. Survey also presents popular benchmark datasets of hate speech/offensive language detection specifying their challenges, the methods for achieving top classification scores, and dataset characteristics such as the number of samples, modalities, language(s), number of classes, etc. Additionally, performance metrics are described, and classification scores of popular hate speech methods are mentioned. The conclusion and future research directions are presented at the end of the survey. Compared with earlier surveys, this paper gives a better presentation of multimodal and multilingual hate speech detection through well-organized comparisons, challenges, and the latest evaluation techniques, along with their best performances.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. M. Bose, “South Asia Journal,” 2020. http://southasiajournal.net/india-senior-bjp-leaders-are-giving-india-a-free-tutorial-in-hate-speech/

  2. R. E. Brannigan, J. L. Moss, and J. Wren, “The conversation,” Fertility and Sterility, 2015. https://theconversation.com/hate-speech-is-still-easy-to-find-on-social-media-106020.

  3. M. Suster, “Business Insider,” Amazon’s Game-Changing Cloud Was Built By Some Guys In South Africa, 2010. https://www.businessinsider.com/736-of-all-statistics-are-made-up-2010-2?r=US&IR=T%0Ahttp://www.businessinsider.com/amazons-game-changing-cloud-was-built-by-some-guys-in-south-africa-2012-3.

  4. A. Schmidt and M. Wiegand, “A Survey on Hate Speech Detection using Natural Language Processing,” Soc. 2017 - 5th Int. Work. Nat. Lang. Process. Soc. Media, Proc. Work. AFNLP SIG Soc., no. 2012, pp. 1–10, 2017, doi: https://doi.org/10.18653/v1/w17-1101.

  5. Cohen-Almagor, R.: Freedom of Expression v. Social Responsibility: Holocaust Denial in Canada. J. Mass Media Ethics Explor. Quest. Media Moral. 28(1), 42–56 (2013). https://doi.org/10.1080/08900523.2012.746119

    Article  Google Scholar 

  6. Delgado, R., Stefancic, J.: Images of the outsider in American law and culture: can free expression remedy deeply inscribed social Ills? Fail. Revolutions 77(6), 3–21 (2019). https://doi.org/10.4324/9780429037627-2

    Article  Google Scholar 

  7. Techterms.com, “Facebook Definition,” 2008. http://www.techterms.com/definition/facebook.

  8. Youtube, “YouTube hate policy,” 2019. https://support.google.com/youtube/answer/2801939?hl=en.

  9. Facebook, “What does facebook consider hate speech?,” 2013. https://www.facebook.com/help/135402139904490.

  10. Nockleby, J.T.: Hate Speech. In: Levy, L.W., Karst, K.L., et al. (eds.) Encyclopedia of the American Constitution, pp. 1277–1279. Macmillan, New York (2000)

    Google Scholar 

  11. Twitter, “Twitter_Hate Definition [online],” 2017. https://support.twitter.com/ articles/.

  12. Davidson, T., Warmsley, D., Macy, M., Webe, I.: Automated hate speech detection and the problem of offensive language. Proc. 11th Int. Conf. Web Soc. Media, ICWSM 11(1), 512–515 (2017)

    Google Scholar 

  13. de Gibert, O., Perez, N., García-Pablos, A., Cuadros, M.: Hate Speech Dataset from a White Supremacy Forum. ar**v preprint ar**v (2019). https://doi.org/10.18653/v1/w18-5102

    Article  Google Scholar 

  14. Fortuna, P., Nunes, S.: A survey on automatic detection of hate speech in text. ACM Comput. Surv. (2018). https://doi.org/10.1145/3232676

    Article  Google Scholar 

  15. Chen, Y., Zhou, Y., Zhu, S., Xu, H.: Detecting offensive language in social media to protect adolescent online safety. Proc. - 2012 ASE/IEEE Int. Conf. Privacy, Secur. Risk Trust 2012 ASE/IEEE Int. Conf. Soc. Comput. Soc (2012). https://doi.org/10.1109/SocialCom-PASSAT.2012.55

    Article  Google Scholar 

  16. Thompson, N.: Equality, Diversity and Social Justice. Sixth, PALGRAVE MACMILLAN (2016)

    Book  Google Scholar 

  17. Guermazi, R., Hammami, M., Ben Hamadou, A.: Using a semi-automatic keyword dictionary for improving violent web site filtering. Proc. - Int. Conf. Signal Image Technol. Internet Based Syst. SITIS (2007). https://doi.org/10.1109/SITIS.2007.137

    Article  Google Scholar 

  18. Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., Chang, Y.: “Abusive language detection in online user content.” 25th Int World Wide Web Conf. WWW 2016, 145–153 (2016). https://doi.org/10.1145/2872427.2883062

    Article  Google Scholar 

  19. Google and Jigsaw, “Perspective API,” 2017. https://perspectiveapi.com.

  20. Asia Centre, “Hate speech in Southeast Asia. New forms, old rules,” 2020. [Online]. Available: https://asiacentre.org/wp-content/uploads/2020/07/Hate-Speech-in-Southeast-Asia-New-Forms-Old-Rules.pdf.

  21. Lomborg, S., Bechmann, A.: Using APIs for data collection on social media. Inf. Soc. 30(4), 256–265 (2014). https://doi.org/10.1080/01972243.2014.915276

    Article  Google Scholar 

  22. Facebook, “Facebook [Online],” 2022. https://www.facebook.com/about/privacy/update.

  23. Lindsey, “Instagrams-Api,” 2022. https://rapidapi.com/blog/how-to-navigate-and-connect-to-instagrams-api/ (accessed Mar. 09, 2022).

  24. Twitter_Rules, “https://help.twitter.com/en/rules-and-policies/twitter-api,” 2022. https://help.twitter.com/en/rules-and-policies/twitter-api.

  25. M. S. Jahan and M. Oussalah, “A systematic review of Hate Speech automatic detection using Natural Language Processing,” ar**v:2106.00742v1, 2021, [Online]. Available: http://arxiv.org/abs/2106.00742.

  26. Dhanya, L.K., Balakrishnan, K.: “Hate speech detection in asian languages: a survey”, ICCISc 2021–2021 Int. Conf. Commun. Control Inf. Sci. Proc. (2021). https://doi.org/10.1109/ICCISc52257.2021.9484922

    Article  Google Scholar 

  27. Poletto, F., Basile, V., Sanguinetti, M., Bosco, C., Patti, V.: Resources and benchmark corpora for hate speech detection: a systematic review. Lang. Resour. Eval. 55(2), 477–523 (2021). https://doi.org/10.1007/s10579-020-09502-8

    Article  Google Scholar 

  28. N. Naaz, Y. Malik, and K. P. Adhiya, “Hate Speech Detection in Twitter-A Survey,” Int. J. Manag. Technol. Eng., vol. 9, no. 1, pp. 1272–1277, 2019, [Online]. Available: http://www.ijamtes.org/gallery/147-jan19.pdf.

  29. Robinson, D., Zhang, Z.: Detection of hate speech in social networks: a survey on multilingual corpus. Comput. Sci. Inf. Technol. (2020). https://doi.org/10.5121/csit.2019.90208

    Article  Google Scholar 

  30. Alrehili, A.: Automatic hate speech detection on social media: A brief survey. Proc. IEEE/ACS Int. Conf. Comput. Syst. Appl. AICCSA (2019). https://doi.org/10.1109/AICCSA47632.2019.9035228

    Article  Google Scholar 

  31. Mohiyaddeen and Dr: Shifaulla Siddiqui, “Automatic hate speech detection: a literature review.” Int. J. Eng. Manag. Res. 11(2), 116–121 (2021). https://doi.org/10.31033/ijemr.11.2.17

    Article  Google Scholar 

  32. Araque, O., Iglesias, C.A.: An Ensemble Method for Radicalization and Hate Speech Detection Online Empowered by Sentic Computing. Cognit. Comput (2022). https://doi.org/10.1007/s12559-021-09845-6

    Article  Google Scholar 

  33. Burnap, P., Williams, M.L.: Us and them: identifying cyber hate on Twitter across multiple protected characteristics. EPJ. Data Sci. (2016). https://doi.org/10.1140/epjds/s13688-016-0072-6

    Article  Google Scholar 

  34. Kwok, I., Wang, Y.: Locate the hate: Detecting tweets against blacks. Proceedings of the 27th AAAI Conference on Artificial Intelligence, AAAI (2013). https://doi.org/10.1609/aaai.v27i1.8539

    Article  Google Scholar 

  35. Baydoğan, V.C., Alatas, B.: Çevrimiçi Sosyal Ağlarda Nefret Söylemi Tespiti için Yapay Zeka Temelli Algoritmaların Performans Değerlendirmesi. Fırat Üniversitesi Mühendislik Bilim. Derg. 33(2), 745–754 (2021). https://doi.org/10.35234/fumbd.986500

    Article  Google Scholar 

  36. Husain, F., Uzuner, O.: “Investigating the Effect of Preprocessing Arabic Text on Offensive Language and Hate Speech Detection”, ACM Trans. Asian Low-Resource Lang. Inf. Process. 21(4), 1–20 (2022). https://doi.org/10.1145/3501398

    Article  Google Scholar 

  37. Chowdhury, A.G.: ARHNet - Leveraging Community Interaction For Detection Of Religious Hate Speech In Arabic”. Proc. 57th Annu. Meet. te Assoc. Comput. Linguist. 2019, 273–280 (2019)

    Article  Google Scholar 

  38. Z. Waseem and D. Hovy, “Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter,” pp. 88–93, 2016, doi: https://doi.org/10.18653/v1/n16-2013

  39. Liu, S., Forss, T.: Combining N-gram based similarity analysis with sentiment analysis in web content classification. KDIR 2014 - Proc. Int. Conf. Knowl. Discov. Inf. Retr (2014). https://doi.org/10.5220/0005170305300537

    Article  Google Scholar 

  40. Greevy, E., Smeaton, A.F.: Classifying racist texts using a support vector machine. Proc. Sheff. SIGIR - Twenty-Seventh Annu. Int. ACM SIGIR Conf. Res. Dev. Inf. Retr (2004). https://doi.org/10.1145/1008992.1009074

    Article  Google Scholar 

  41. Badjatiya, P., Gupta, S., Gupta, M., Varma, V.: Deep learning for hate speech detection in tweets. 26th Int. World Wide Web Conf. 2017, WWW 2017 Companion 2, 759–760 (2017). https://doi.org/10.1145/3041021.3054223

    Article  Google Scholar 

  42. Katona, E., Buda, J., Bolonyai, F.: Using N-grams and Statistical Features to Identify Hate Speech Spreaders on Twitter. CEUR Workshop Proc. 2021, 2025–2034 (2021)

    Google Scholar 

  43. Mehdad, Y., Tetreault, J.: “Do Characters Abuse More Than Words? Dialogue. SIGDIAL 2016 - 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference (2016). https://doi.org/10.18653/v1/w16-3638

    Article  Google Scholar 

  44. Mulki, H., Ali, C.B., Haddad, H., Babao, I.: Tw-StAR at SemEval-2019 Task 5: N-gram embeddings for Hate Speech Detection in Multilingual Tweets. Proc. 13th Int Work. Semant. Eval. 2019, 503–507 (2019)

    Google Scholar 

  45. S. N. Group, “Stanford NLP Group,” 2005. https://nlp.stanford.edu/.

  46. Wang, C., Day, M., Wu, C.: Political Hate Speech Detection and Lexicon Building : A Study in Taiwan. IEEE Access 10, 44337–44346 (2022). https://doi.org/10.1109/ACCESS.2022.3160712

    Article  Google Scholar 

  47. Liu, S., Forss, T.: “New classification models for detecting hate and violence web content Knowl. IC3K 2015 - Proc. 7th Int. Jt. Conf. Knowl. Discov. Knowl. Eng. Knowl. Manag 1, 487–495 (2015). https://doi.org/10.5220/0005636704870495

    Article  Google Scholar 

  48. Gitari, N.D., Zu**, Z., Damien, H., Long, J.: A lexicon-based approach for hate speech detection. Int. J. Multimed. Ubiquitous Eng. 10(4), 215–230 (2015). https://doi.org/10.14257/ijmue.2015.10.4.21

    Article  Google Scholar 

  49. S. Agarwal and A. Sureka, “Characterizing Linguistic Attributes for Automatic Classification of Intent Based Racist/Radicalized Posts on Tumblr Micro-Blogging Website,” 2017, [Online]. Available: http://arxiv.org/abs/1701.04931.

  50. Del Vigna, F., Cimino, A., Dell’Orletta, F., Petrocchi, M., Tesconi, M.: Hate me, hate me not: Hate speech detection on Facebook. CEUR Workshop Proc. 1816, 86–95 (2017)

    Google Scholar 

  51. Ali, M.Z., Rauf, S., Javed, K., Hussain, S.: Improving hate speech detection of urdu tweets using sentiment analysis. IEEE Access 9, 84296–84305 (2021). https://doi.org/10.1109/ACCESS.2021.3087827

    Article  Google Scholar 

  52. Baydogan, C., Alatas, B.: Sentiment analysis in social networks using social spider optimization algorithm. Teh. Vjesn. 28(6), 1943–1951 (2021). https://doi.org/10.17559/TV-20200614172445

    Article  Google Scholar 

  53. Pablo, J., Jiménez, J.: Topic modelling of racist and xenophobic YouTube comments. Analyzing hate speech against migrants and refugees spread through YouTube in Spanish. TEEM’21 Ninth Int. Conf. Technol. Ecosyst. Enhancing Multicult 2021, 456–460 (2021)

    Google Scholar 

  54. Liu, H., Alorainy, W., Burnap, P., Williams, M.L.: Fuzzy multi-task learning for hate speech type identification. Web Conf. 2019 - Proc. World Wide Web Conf. WWW (2019). https://doi.org/10.1145/3308558.3313546

    Article  Google Scholar 

  55. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3(4–5), 993–1022 (2003). https://doi.org/10.1016/b978-0-12-411519-4.00006-9

    Article  MATH  Google Scholar 

  56. V. Mujadia, “IIIT-Hyderabad at HASOC 2019 : Hate Speech Detection,” CEUR Workshop Proc., 2019.

  57. Kumar, P.M.A., Pradesh, A.: Hate Speech Detection using Text and Image Tweets Based On Bi-directional Long Short-Term Memory. 2021 Int. Conf. Disruptive Technol. Multi-Disciplinary Res. Appl 2021, 158–162 (2021)

    Article  Google Scholar 

  58. Dinakar, K., Reichart, R., Lieberman, H.: Modeling the detection of textual cyberbullying. AAAI Work. - Tech. Rep. 11–02, 11–17 (2011)

    Google Scholar 

  59. Gaydhani, A., Doma, V., Kendre, S., Bhagwat, L.: Detecting Hate Speech and Offensive Language on Twitter using Machine Learning : An N-gram and TFIDF based Approach. IEEE Int. Adv. Comput. Conf 1809, 08651 (2018)

    Google Scholar 

  60. Gambino, G., Pirrone, R., Ingegneria, D.: CHILab @ HaSpeeDe 2: Enhancing Hate Speech Detection with Part-of-Speech Tagging. CEUR Workshop Proc. 2020, 165 (2020)

    Google Scholar 

  61. Erizal, E., Setianingsih, C.: “Hate Speech Detection in Indonesian Language on Instagram Comment Section Using Maximum Entropy Classification Method.” 2019 Int Conf. Inf. Commun. Technol. 2019, 533–538 (2019)

    Google Scholar 

  62. Bilal, M., Khan, A., Jan, S., Musa, S.: Context-Aware Deep Learning Model for Detection of Roman Urdu Hate Speech on Social Media Platform. IEEE Access 10, 121133–121151 (2022). https://doi.org/10.1109/ACCESS.2022.3216375

    Article  Google Scholar 

  63. Zhou, X., et al.: “Hate Speech Detection based on Sentiment Knowledge Sharing.” Proc. 59th Annu. Meet. Assoc. Comput. Linguist. 11th Int Jt. Conf. Nat. Lang. Process. 2021, 7158–7166 (2021)

    Google Scholar 

  64. Plaza-del-Arco, F.M., Molina-González, M.D., Ureña-López, L.A., Martín-Valdivia, M.T.: Comparing pre-trained language models for Spanish hate speech detection. Expert Syst. Appl. 166, 114120 (2021). https://doi.org/10.1016/j.eswa.2020.114120

    Article  Google Scholar 

  65. W. Warner and J. Hirschberg, “Detecting hate speech on the world wide web,” in Proceeding LSM ’12 Proceedings of the Second Workshop on Language in Social Media, 2012, no. Lsm, pp. 19–26, [Online]. Available: http://dl.acm.org/citation.cfm?id=2390374.2390377.

  66. Haralambous, Y., Lenca, P.: Text classification using association rules, dependency pruning and hyperonymization. CEUR Workshop Proc. 1202, 65–80 (2014)

    Google Scholar 

  67. Abro, S., Shaikh, S., Ali, Z.: Automatic Hate Speech Detection using Machine Learning : A Comparative Study. Int. J. Adv. Comput. Sci. App. 11(8), 484–491 (2020)

    Google Scholar 

  68. Baydogan, C., Alatas, B.: Deep-Cov19-Hate: A Textual-Based Novel Approach for Automatic Detection of Hate Speech in Online Social Networks throughout COVID-19 with Shallow and Deep Learning Models. Teh. Vjesn. 29(1), 149–156 (2022). https://doi.org/10.17559/TV-20210708143535

    Article  Google Scholar 

  69. Chiril, P., Wahyu, E., Farah, P., Véronique, B., Viviana, M., Patti, V.: Emotionally Informed Hate Speech Detection : A Multi - target Perspective. Cognit. Comput. (2022). https://doi.org/10.1007/s12559-021-09862-5

    Article  Google Scholar 

  70. Mullah, N.S., Zainon, W.M.N.W.: Advances in machine learning algorithms for hate speech detection in social media: a review. IEEE Access 9, 88364–88376 (2021). https://doi.org/10.1109/ACCESS.2021.3089515

    Article  Google Scholar 

  71. Naseem, U., Razzak, I., Eklund, P.W.: “A survey of pre-processing techniques to improve short-text quality : a case study on hate speech detection on twitter. Multimed Tools Appl (2020). https://doi.org/10.1007/s11042-020-10082-6

    Article  Google Scholar 

  72. P. Burnap and M. Williams, “Hate Speech, Machine Classification and Statistical Modelling of Information Flows on Twitter: Interpretation and Communication for Policy Decision Making,” in Internet, Policy & Politics, 2014, pp. 1–18, [Online]. Available: http://orca.cf.ac.uk/id/eprint/65227%0A.

  73. Khan, M.M., Shahzad, K., Malik, M.K.: “Hate speech detection in Roman Urdu”, ACM Trans. Asian Low-Resource Lang. Inf. Process. 20(1), 1–19 (2021). https://doi.org/10.1145/3414524

    Article  Google Scholar 

  74. Hua, T., Chen, F., Zhao, L., Lu, C.-T., Ramakrishnan, N.: STED: semi-supervised targeted-interest event detection”. Knowledge Discov. Data Mining 2013, 1466–1469 (2013)

    Google Scholar 

  75. Ali, R., Farooq, U., Arshad, U., Shahzad, W., Omer, M.: Computer speech & language hate speech detection on Twitter using transfer learning. Comput. Speech Lang. 74, 101365 (2022). https://doi.org/10.1016/j.csl.2022.101365

    Article  Google Scholar 

  76. Ma, C., Du, X., Cao, L.: Improved KNN algorithm for fine-grained classification of encrypted network flow. Mdpi Electron (2020). https://doi.org/10.3390/electronics9020324

    Article  Google Scholar 

  77. Ferreira, P.J.S., Cardoso, J.M.P.: k NN prototy** schemes for embedded human activity recognition with online learning. Mdpi Comput (2020). https://doi.org/10.3390/computers9040096

    Article  Google Scholar 

  78. Kumar, P., Bhawal, S.: Computer speech & language hate speech and offensive language detection in Dravidian languages using deep ensemble framework. Comput. Speech Lang. (2022). https://doi.org/10.1016/j.csl.2022.101386

    Article  Google Scholar 

  79. Alfina, I., Mulia, R., Fanany, M.I., Ekanata, Y.: 2018 “Hate speech detection in the Indonesian language: a dataset and preliminary study. 2017 Int. Conf. Adv. Comput. Sci. Inf. Syst. ICACSIS 2018, 233–237 (2017). https://doi.org/10.1109/ICACSIS.2017.8355039

    Article  Google Scholar 

  80. Bosco, C., Orletta, F.D., Poletto, F., Tesconi, M.: Overview of the EVALITA 2018 hate speech detection task. CEUR Workshop Proc. (2018). https://doi.org/10.4000/books.aaccademia.4503

    Article  Google Scholar 

  81. Bai, X., Merenda, F., Zaghi, C., Caselli, T., Nissim, M.: RuG EVALITA 2018: hate speech detection in Italian social media. CEUR Workshop Proc. 2263, 1–5 (2018)

    Google Scholar 

  82. Chen, H., McKeever, S., Delany, S.J.: Abusive text detection using neural networks. CEUR Workshop Proceedings, 2086(2), 258–260.ction using neural networks. CEUR Workshop Proc. 2086(2), 258–260 (2017)

    Google Scholar 

  83. M. Wiegand, J. Ruppenhofer, A. Schmidt, and C. Greenberg, “Inducing a lexicon of abusive words ? a feature-based approach,” NAACL HLT 2018 - 2018 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. - Proc. Conf., vol. 1, pp. 1046–1056, 2018, doi: https://doi.org/10.18653/v1/n18-1095.

  84. Pawar, R., Agrawal, Y., Joshi, A., Gorrepati, R., Raje, R.R.: “Cyberbullying detection system with multiple server configurations. IEEE Int. Conf. Electro Inf. Technol. (2018). https://doi.org/10.1109/EIT.2018.8500110

    Article  Google Scholar 

  85. N. Djuric, J. Zhou, R. Morris, M. Grbovic, V. Radosavljevic, and N. Bhamidipati, “Hate speech detection with comment embeddings,” in WWW 2015 companion—proceedings of the 24th international conference on World Wide Web, 2015, pp. 29–30, doi: https://doi.org/10.1145/2740908.2742760.

  86. W. Z, “Are you a racist or am i seeing things? Annotator influence on hate speech detection on twitter.,” in proceedings of the first workshop on NLP and computational social science. Association for computational linguistics, 2016, pp. 138–142.

  87. Malmasi, S., Zampieri, M.: Detecting hate speech in social media. Int. Conf. Recent. Adv. Nat. Lang. Process. RANLP. 2017, 467–472 (2017). https://doi.org/10.26615/978-954-452-049-6-062

    Article  Google Scholar 

  88. M. R. Jha A, “When does a compliment become sexist? analysis and classification of ambivalent sexism using twitter data. In:,” In proceedings of the second workshop on NLP and computational social science. Association for Computational Linguistics, 2017, pp. 7–16.

  89. Santosh, T.Y.S.S., Aravind, K.V.S.: Hate speech detection in Hindi-English code-mixed social media text. ACM Int. Conf. Proc. Ser. (2019). https://doi.org/10.1145/3297001.3297048

    Article  Google Scholar 

  90. Özel, S.A., Akdemir, S., Saraç, E., Aksu, H.: “Detection of cyberbullying on social media messages in Turkish”, 2nd Int. Conf. Comput. Sci. Eng. UBMK 2017, 366–370 (2017). https://doi.org/10.1109/UBMK.2017.8093411

    Article  Google Scholar 

  91. M. Fernandez and H. Alani, “Contextual semantics for radicalisation detection on Twitter,” CEUR Workshop Proc., vol. 2182, 2018.

  92. Abozinadah, E.A., Mbaziira, A.V., Jones, J.H.J.: Detection of abusive accounts with Arabic tweets. Int. J. Knowl. Eng. 1(2), 113–119 (2015). https://doi.org/10.7763/ijke.2015.v1.19

    Article  Google Scholar 

  93. Abozinadah, E.A., Jones, J.H.: A statistical learning approach to detect abusive twitter accounts. ACM Int. Conf. Proceeding Ser (2017). https://doi.org/10.1145/30932413093281

    Article  Google Scholar 

  94. Alakrot, A., Murray, L., Nikolov, N.S.: Dataset construction for the detection of anti-social behaviour in online communication in Arabic. Procedia Comput. Sci. 142, 174–181 (2018). https://doi.org/10.1016/j.procs.2018.10.473

    Article  Google Scholar 

  95. Alakrot, A., Murray, L., Nikolov, N.S.: Towards accurate detection of offensive language in online communication in Arabic. Procedia Comput. Sci. 142, 315–320 (2018). https://doi.org/10.1016/j.procs.2018.10.491

    Article  Google Scholar 

  96. A. A. E. M. B. N. H. Alhuzali and M. Abdul-Mageed, “Think Before Your Click: Data and Models for Adult Content in Arabic Twitter,” Proc. Elev. Int. Conf. Lang. Resour. Eval. (LREC 2018), 2018.

  97. Haidar, B., Chamoun, M., Serhrouchni, A.: A multilingual system for cyberbullying detection: Arabic content detection using machine learning. Adv. Sci. Technol. Eng. Syst. 2(6), 275–284 (2017). https://doi.org/10.25046/aj020634

    Article  Google Scholar 

  98. Magdy, W., Darwish, K., Weber, I.: Failed revolutions: using Twitter to study the antecedents of ISIS support. First Monday (2016). https://doi.org/10.5210/fm.v21i2.6372

    Article  Google Scholar 

  99. **ang, G., Fan, B., Wang, L., Hong, J., Rose, C.: “Detecting offensive tweets via topical feature discovery over a large scale twitter corpus. ACM Int. Conf. Proceeding Ser (2012). https://doi.org/10.1145/23967612398556

    Article  Google Scholar 

  100. V. Nahar, S. Al-maskari, X. Li, and C. Pang, “Databases Theory and Applications - 25th Australasian Database Conference, {ADC} 2014, Brisbane, QLD, Australia, July 14–16, 2014. Proceedings,” vol. 8506, 2019, 2014, doi: https://doi.org/10.1007/978-3-319-08608-8.

  101. Agarwal, S., Sureka, A.: ‘Using KNN and SVM based one-class classifier for detecting online radicalization on twitter.’ Int. Conf. Distributed Comput. Internet Technol. (2015). https://doi.org/10.1007/978-3-319-14977-6_47

    Article  Google Scholar 

  102. Kaati, L., Omer, E., Prucha, N., Shrestha, A.: Detecting multipliers of Jihadism on Twitter. Proc 15th IEEE Int. Conf. Data Min. Work. ICDMW (2015). https://doi.org/10.1109/ICDMW.2015.9

    Article  Google Scholar 

  103. Di Capua, M., Di Nardo, E., Petrosino, A.: Unsupervised cyber bullying detection in social networks. Proc. Int. Conf. Pattern Recognit. (2016). https://doi.org/10.1109/ICPR.2016.7899672

    Article  Google Scholar 

  104. Abdelfatah, K.E., Terejanu, G., Alhelbawy, A.A.: “Unsupervised Detection of Violent Content in Arabic Social Media. Comput. Sci.Info Technol. (2017). https://doi.org/10.5121/csit.2017.70401

    Article  Google Scholar 

  105. Pitsilis, G.K., Ramampiaro, H., Langseth, H.: Effective hate-speech detection in Twitter data using recurrent neural networks. Appl. Intell. 48(12), 4730–4742 (2018). https://doi.org/10.1007/s10489-018-1242-y

    Article  Google Scholar 

  106. S. Suryawanshi, B. R. Chakravarthi, M. Arcan, and P. Buitelaar, “Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text,” Proc. Second Work. Trolling, Aggress. Cyberbullying, vol. 2020-Decem, no. May, pp. 32–41, 2020, [Online]. Available: https://www.aclweb.org/anthology/2020.trac-1.6.

  107. T. Deshpande and N. Mani, An Interpretable Approach to Hateful Meme Detection, vol. 1, no. 1. Association for Computing Machinery, 2021.

  108. D. Kiela et al., “The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes,” pp. 1–17, 2020, [Online]. Available: http://arxiv.org/abs/2005.04790.

  109. N. Muennighoff, “Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memes,” ar**v:2012.07788v1, pp. 1–6, 2020, [Online]. Available: http://arxiv.org/abs/2012.07788.

  110. Yuan, S., Wu, X., **ang, Y.: A two phase deep learning model for identifying discrimination from tweets. In Adv. Database Technol. EDBT (2016). https://doi.org/10.5441/002/edbt.2016.92

    Article  Google Scholar 

  111. Gambäck, B., Sikdar, U.K.: “Using Convolutional Neural Networks to Classify Hate-Speech. In Proceed First Workshop Abusive Language Online (2017). https://doi.org/10.18653/v1/w17-3013

    Article  Google Scholar 

  112. F. P. Park JH, “One-step and two-step classification for abusive language detection on twitter.,” 2017.

  113. Zhang, Z., Robinson, D., Tepper, J.: “Detecting hate speech on twitter using a convolution gru based deep neural network. In Lecture Note. Comput. Sci. 10843, 745–760 (2018)

    Article  Google Scholar 

  114. Erico, C., Salim, R., Suhartono, D.: A systematic literature review of different machine learning methods on hate speech detection. Int. J. Info. Vis. 4, 213–218 (2020)

    Google Scholar 

  115. Ayo, F.E., Folorunso, O., Ibharalu, F.T., Osinuga, I.A., Abayomi-Alli, A.: “A probabilistic clustering model for hate speech classification in twitter. Expert Syst. Appl. (2021). https://doi.org/10.1016/j.eswa.2021.114762

    Article  Google Scholar 

  116. Baydogan, C., Alatas, B.: Metaheuristic ant lion and moth flame optimization-based novel approach for automatic detection of hate speech in online social networks. IEEE Access 9, 110047–110062 (2021). https://doi.org/10.1109/ACCESS.2021.3102277

    Article  Google Scholar 

  117. Asiri, Y., Halawani, H.T., Alghamdi, H.M., Abdalaha Hamza, S.H., Abdel-Khalek, S., Mansour, R.F.: enhanced seagull optimization with natural language processing based hate speech detection and classification. Appl. Sci. (2022). https://doi.org/10.3390/app12168000

    Article  Google Scholar 

  118. Y. G. and X. L. Pengfei Du,: Towards an intrinsic interpretability approach for multimodal hate speech detection. Int. J. Pattern Recognit. Artif. Intell. (2022). https://doi.org/10.1142/S0218001422500409

    Article  Google Scholar 

  119. N. Albadi, M. Kurdi, and S. Mishra 2018 “Are they our brothers? Analysis and detection of religious hate speech in the Arabic Twittersphere. Proc. 2018 IEEE/ACM Int. Conf. Adv. Soc. Networks Anal. Mining, ASONAM. Doi: https://doi.org/10.1109/ASONAM.2018.8508247.

  120. Miok, K., Škrlj, B., Zaharie, D., Robnik-Šikonja, M.: To ban or not to ban: bayesian attention networks for reliable hate speech detection. Cognit. Comput. 14(1), 353–371 (2022). https://doi.org/10.1007/s12559-021-09826-9

    Article  Google Scholar 

  121. Wullach, T., Adler, A., Minkov, E.: Towards hate speech detection at large via deep generative modeling. IEEE Internet Comput. 25(2), 48–57 (2021). https://doi.org/10.1109/MIC.2020.3033161

    Article  Google Scholar 

  122. Gomez, R., Gibert, J., Gomez, L., Karatzas, D.: “Exploring hate speech detection in multimodal publications. Proc. - 2020 IEEE Winter Conf. Appl. Comput. Vision, WACV (2020). https://doi.org/10.1109/WACV45572.2020.9093414

    Article  Google Scholar 

  123. A. Das, J. S. Wahi, and S. Li, “Detecting Hate Speech in Multi-modal Memes,” 2020, [Online]. Available: http://arxiv.org/abs/2012.14891.

  124. Zhou, Y., Yang, Y., Liu, H., Liu, X., Savage, N.: Deep learning based fusion approach for hate speech detection. IEEE Access 8, 128923–128929 (2020). https://doi.org/10.1109/ACCESS.2020.3009244

    Article  Google Scholar 

  125. Muhammad, I.Z., Nasrun, M., Setianingsih, C.: Hate speech detection using global vector and deep belief network algorithm. 2020 1st Int. Conf. Big Data Anal. Pract. IBDAP (2020). https://doi.org/10.1109/IBDAP50342.2020.9245467

    Article  Google Scholar 

  126. Le-hong, P.: Knowledge-based systems diacritics generation and application in hate speech detection on vietnamese social networks. Knowledge-Based Syst. (2021). https://doi.org/10.1016/j.knosys.2021.107504

    Article  Google Scholar 

  127. P. Vijayaraghavan, H. Larochelle, and D. Roy, “Interpretable Multi-Modal Hate Speech Detection,” Int. Conf. Mach. Learn., 2021.

  128. G. Sahu, R. Cohen, and O. Vechtomova, “Towards A Multi-agent System for Online Hate Speech Detection,” Proc. 20th Int. Conf. Auton. Agents Multiagent Syst., 2021.

  129. A. Jiang, Aiqi; Zubiaga, Cross-lingual Capsule Network for Hate Speech Detection in Social Media, vol. 1, no. 1. Association for Computing Machinery, 2021.

  130. Perifanos, K.: Multimodal hate speech detection in greek social media. Mdpi Multimed. Technol. Interact. (2021). https://doi.org/10.3390/mti5070034

    Article  Google Scholar 

  131. Aldjanabi, W., Dahou, A., Al-qaness, M.A.A., Elaziz, M.A., Helmi, A.M., Damaševi, R.: Arabic offensive and hate speech detection using a cross-corpora multi-task learning model. Mdpi inform. (2021). https://doi.org/10.3390/informatics8040069

    Article  Google Scholar 

  132. Al-Makhadmeh, Z., Tolba, A.: Automatic hate speech detection using killer natural language processing optimizing ensemble deep learning approach. Computing 102(2), 501–522 (2020). https://doi.org/10.1007/s00607-019-00745-0

    Article  Google Scholar 

  133. Corazza, M., Menini, S., Cabrio, E., Tonelli, S., Villata, S.: A multilingual evaluation for online hate speech detection. ACM Trans. Internet Technol. (2020). https://doi.org/10.1145/3377323

    Article  Google Scholar 

  134. Kapil, P., Ekbal, A.: A deep neural network based multi-task learning approach to hate speech detection. Knowledge-Based Syst. (2020). https://doi.org/10.1016/j.knosys.2020.106458

    Article  Google Scholar 

  135. Aulia, N., Budi, I.: Hate speech detection on Indonesian long text documents using machine learning approach. ACM Int. Conf. Proceeding Ser. (2019). https://doi.org/10.1145/33304823330491

    Article  Google Scholar 

  136. Badjatiya, P., Gupta, M., Varma, V.: Stereotypical bias removal for hate speech detection task using knowledge-based generalizations. Web Conf. 2019 - Proc World Wide Web Conf. WWW 10(1145/3308558), 3313504 (2019)

    Google Scholar 

  137. G. Nascimento, F. Carvalho, A. M. Da Cunha, C. R. Viana, and G. P. Guedes, 2019 “Hate speech detection using Brazilian imageboards,” Proc. 25th Brazillian Symp. Multimed. Web, WebMedia. https://doi.org/10.1145/3323503.3360619.

  138. A. S. Saksesi, M. Nasrun, and C. Setianingsih, “Analysis Text of Hate Speech Detection Using Recurrent Neural Network,” Proc. - 2018 Int. Conf. Control. Electron. Renew. Energy Commun. ICCEREC 2018, pp. 242–248, 2018, doi: https://doi.org/10.1109/ICCEREC.2018.8712104.

  139. Mozafari, M., Farahbakhsh, R., Crespi, N.: A BERT-Based transfer learning approach for hate speech detection in online social media. Conf. Comp. Net, Their Appl Int (2020). https://doi.org/10.1007/978-3-030-36687-2_77

    Article  Google Scholar 

  140. V. Basile et al., “SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in Twitter,” NAACL HLT 2019 - Int. Work. Semant. Eval. SemEval 2019, Proc. 13th Work., pp. 54–63, 2019, doi: https://doi.org/10.18653/v1/s19-2007.

  141. N. Mehrabi, F. Morstatter, N. Saxena, and L. G. Jan, “A Survey on Bias and Fairness in Machine Learning,” ar**v:1908.09635v3, 2022.

  142. Ahmed, Z., Vidgen, B., Hale, S.A.: Tackling racial bias in automated online hate detection : towards fair and accurate detection of hateful users with geometric deep learning. EPJ Data Sci. (2022). https://doi.org/10.1140/epjds/s13688-022-00319-9

    Article  Google Scholar 

  143. Watanabe, H., Bouazizi, M., Ohtsuki, T.: Hate speech on twitter: a pragmatic approach to collect hateful and offensive expressions and perform hate speech detection. IEEE Access 6, 13825–13835 (2018). https://doi.org/10.1109/ACCESS.2018.2806394

    Article  Google Scholar 

  144. Mollas, I., Chrysopoulou, Z., Karlos, S., Tsoumakas, G.: ETHOS: a multi-label hate speech detection dataset. Complex Intell. Syst. (2022). https://doi.org/10.1007/s40747-021-00608-2

    Article  Google Scholar 

  145. A. Velankar, H. Patil, A. Gore, S. Salunke, and R. Joshi, “L3Cube-MahaHate: A Tweet-based Marathi Hate Speech Detection Dataset and BERT models,” ar**v:2203.13778v2, pp. 1–12, 2022, [Online]. Available: http://arxiv.org/abs/2203.13778.

  146. Mathew, B., Saha, P., Yimam, S.M., Biemann, C., Goyal, P., Mukherjee, A.: “HateXplain: a benchmark dataset for explainable hate speech detection”,. 35th aaai conf Artif. Intell. AAAI 17A, 14867–14875 (2021)

    Google Scholar 

  147. K. Yang, W. Jang, and W. I. Cho, “APEACH: Attacking Pejorative Expressions with Analysis on Crowd-Generated Hate Speech Evaluation Datasets,” 2022, [Online]. Available: http://arxiv.org/abs/2202.12459.

  148. J. A. Leite, D. F. Silva, K. Bontcheva, and C. Scarton, “Toxic Language Detection in Social Media for Brazilian Portuguese: New Dataset and Multilingual Analysis,” ar**v:2010.04543, 2020, [Online]. Available: http://arxiv.org/abs/2010.04543.

  149. C. S. de Alcântara, D. Feijó, and V. P. Moreira, “Offensive video detection: Dataset and baseline results,” Lr. 2020 - 12th Int. Conf. Lang. Resour. Eval. Conf. Proc., no. May, pp. 4309–4319, 2020.

  150. T. Hartvigsen, S. Gabriel, H. Palangi, M. Sap, D. Ray, and E. Kamar, “ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection,” 2022, [Online]. Available: http://arxiv.org/abs/2203.09509.

  151. Evkoski, B., Pelicon, A., Mozetič, I., Ljubešić, N., Novak, P.K.: Retweet communities reveal the main sources of hate speech. PLoS ONE (2022). https://doi.org/10.1371/journal.pone.0265602

    Article  Google Scholar 

  152. H. R. Kirk, B. Vidgen, P. Röttger, T. Thrush, and S. A. Hale, “Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hate,” 2021, [Online]. Available: http://arxiv.org/abs/2108.05921.

  153. N. Romim, M. Ahmed, M. S. Islam, A. Sen Sharma, H. Talukder, and M. R. Amin, “HS-BAN: A Benchmark Dataset of Social Media Comments for Hate Speech Detection in Bangla,” ar**v:2112.01902v1, pp. 1–8, 2021, [Online]. Available: http://arxiv.org/abs/2112.01902.

  154. M. R. Karim, B. R. Chakravarthi, J. P. McCrae, and M. Cochez,2020 “Classification Benchmarks for Under-resourced Bengali Language based on Multichannel Convolutional-LSTM Network. Proc. - 2020 IEEE 7th Int. Conf. Data Sci. Adv. Anal. DSAA. https://doi.org/10.1109/DSAA49011.2020.00053.

  155. Wu, C.S., Bhandary, U.: Detection of hate speech in videos using machine learning. Int. Conf. Comput. Sci. Comput. Intell. CSCI Proc (2020). https://doi.org/10.1109/CSCI51800.2020.00104

    Article  Google Scholar 

  156. M. Zampieri et al., “SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020),” Proc. Int. Work. Semant. Eval., no. OffensEval, 2020.

  157. Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., Kumar, R.: Predicting the type and target of offensive posts in social media. Proc. NAACL-HLT 2019, 1415–1420 (2019)

    Google Scholar 

  158. Haddad, H., Mulki, H., Oueslati, A.: T-HSAB: a tunisian hate speech and abusive dataset. Commun. Comput. Inf. Sci. (2019). https://doi.org/10.1007/978-3-030-32959-4_18

    Article  Google Scholar 

  159. H. Mulki, H. Haddad, C. Bechikh Ali, and H. Alshabani, “L-HSAB: A Levantine Twitter Dataset for Hate Speech and Abusive Language,” pp. 111–118, 2019, doi: https://doi.org/10.18653/v1/w19-3512.

  160. Elsherief, M., Nilizadeh, S., Nguyen, D., Vigna, G., Belding, E.: Peer to peer hate: hate speech instigators and their targets. AAAI Conf. Web Soc. Media (2018). https://doi.org/10.1609/icwsm.v12i1.15038

    Article  Google Scholar 

  161. A. M. Founta et al., “Large scale crowdsourcing and characterization of twitter abusive behavior,” 12th Int. AAAI Conf. Web Soc. Media, ICWSM 2018, no. Icwsm, pp. 491–500, 2018

  162. J. Moon, W. I. Cho, and J. Lee, “BEEP ! Korean Corpus of Online News Comments for Toxic Speech Detection,” pp. 25–31, 2017.

  163. H. Mubarak, K. Darwish, and W. Magdy, “Abusive Language Detection on Arabic Social Media,” Proc. First Work. Abus. Lang. Online, pp. 52–56, 2017, doi: https://doi.org/10.18653/v1/w17-3008.

  164. Toutenburg, H.: Mathematical statistics with applications. Computational Statistics & Data Anal (1992). https://doi.org/10.1016/0167-9473(92)90162-9

    Article  MathSciNet  Google Scholar 

  165. C. A. Goodfellow I, Bengio Y, Deep Learning. MIT Press, Cambridge, 2016.

  166. T. Mikolov, A. Deoras, S. Kombrink, L. Burget, and J. H. Černocký, “Empirical evaluation and combination of advanced language modeling techniques,” Proc. Annu. Conf. Int. Speech Commun. Assoc. INTERSPEECH, no. August, pp. 605–608, 2011.

  167. De Souza, G.A., Da Costa-Abreu, M.: Automatic offensive language detection from Twitter data using machine learning and feature selection of metadata. Proc. Int. Jt. Conf. Neural Networks (2020). https://doi.org/10.1109/IJCNN48605.2020.9207652

    Article  Google Scholar 

  168. M. Polignano et al., “A L BERT O : Italian BERT Language Understanding Model for NLP Challenging Tasks Based on Tweets,” CEUR Workshop Proc., 2019.

  169. Webb, G.I.: Decision tree grafting from the all-tests-but-one partition. IJCAI Int. Jt. Conf. Artif. Intell. 2, 702–707 (1999)

    Google Scholar 

  170. S. Tulkens, L. Hilte, E. Lodewyckx, B. Verhoeven, and W. Daelemans, “A Dictionary-based Approach to Racism Detection in Dutch Social Media,” 2016, [Online]. Available: http://arxiv.org/abs/1608.08738.

  171. Burnap, P., Williams, M.L.: Cyber hate speech on twitter: an application of machine classification and statistical modeling for policy and decision making. Policy Internet 7(2), 223–242 (2015). https://doi.org/10.1002/poi3.85

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Biometric Research Laboratory, Department of Information Technology, Delhi Technological University, Delhi, 110042, India

    Anusha Chhabra & Dinesh Kumar Vishwakarma

Authors
  1. Anusha Chhabra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dinesh Kumar Vishwakarma
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Anusha Chhabra: Software, Validation, Investigation, Data Curation, Writing – Original Draft, Visualization. Dinesh Kumar Vishwakarma: Conceptualization, Methodology, Formal Analysis, Resources, Writing – Review & Editing, Supervision, Project Administration, Funding Acquisition.

Corresponding author

Correspondence to Dinesh Kumar Vishwakarma.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chhabra, A., Vishwakarma, D.K. A literature survey on multimodal and multilingual automatic hate speech identification. Multimedia Systems 29, 1203–1230 (2023). https://doi.org/10.1007/s00530-023-01051-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-023-01051-8

Keywords

Search

Navigation