SpringerLink
Log in

MMOD-MEME: A Dataset for Multimodal Face Emotion Recognition on Code-Mixed Tamil Memes

  • Conference paper
  • First Online:
Speech and Language Technologies for Low-Resource Languages (SPELLL 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1802))

  • 235 Accesses

Abstract

Multimodal Facial Emotion Recognition (MFER) for Low resourced Language like Tamil is handled with code-mixed text of Tamil and English. The newly created dataset addresses the multimodal approach on facial emotion recognition with the help of code-mixed memes. The dataset provides facial emotions for the memes and the code-mixed comments for the memes. The memes posted in websites and social media are collected to prepare the dataset. Overall dataset contains 4962 memes with annotated facial emotions and the code-mixed memes in it. Each are annotated by the 3 different annotators with single face emotion and double face emotions with code-mixed Tamil memes. Convolutional Neural Network (CNN) has been applied for detecting the emotions on this dataset containing single faces alone. The preliminary results on the single face emotion dataset has resulted in an accuracy of 0.3028.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Germany)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 67.40
Price includes VAT (Germany)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 85.59
Price includes VAT (Germany)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. https://www.smartinsights.com/social-media-marketing/social-media-strategy/new-global-social-media-research/#: :text=more%20than%20half%20of%20the, social%20media%20is%202h%2029m

  2. Caridakis, G., et al.: Multimodal emotion recognition from expressive faces, body gestures and speech. In: Boukis, C., Pnevmatikakis, A., Polymenakos, L. (eds.) AIAI 2007. ITIFIP, vol. 247, pp. 375–388. Springer, Boston, MA (2007). https://doi.org/10.1007/978-0-387-74161-1_41

    Chapter  Google Scholar 

  3. Ganganwar, V., Rajalakshmi, R.: Implicit aspect extraction for sentiment analysis: a survey of recent approaches. Procedia Comput. Sci. 165, 485–491 (2019)

    Article  Google Scholar 

  4. Ganganwar, V., Rajalakshmi, R.: MTDOT: a multilingual translation-based data augmentation technique for offensive content identification in Tamil text data. Electronics 11(21) (2022). https://doi.org/10.3390/electronics11213574. https://www.mdpi.com/2079-9292/11/21/3574

  5. Hirat, R., Mittal, N.: A survey on emotion detection techniques using text in blogposts (2015)

    Google Scholar 

  6. Kannan, R.R., Rajalakshmi, R., Kumar, L.: IndicBERT based approach for sentiment analysis on code-mixed tamil tweets 3159, 8 (2021)

    Google Scholar 

  7. Li, M., et al.: Multimodal emotion recognition model based on a deep neural network with multiobjective optimization. Wirel. Commun. Mob. Comput. 2021, 6971100:1–6971100:10 (2021)

    Google Scholar 

  8. Liu, D., Wang, Z., Wang, L., Chen, L.: Multi-modal fusion emotion recognition method of speech expression based on deep learning. Front. Neurorobotics 15 (2021). https://doi.org/10.3389/fnbot.2021.697634. https://www.frontiersin.org/articles/10.3389/fnbot.2021.697634

  9. R., R.: Supervised term weighting methods for url classification. J. Comput. Sci. 10 (06 2014). https://doi.org/10.3844/jcssp.2014.1969.1976

  10. R., R., Aravindan, C.: An effective and discriminative feature learning for url based web page classification. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 1374–1379 (2018). https://doi.org/10.1109/SMC.2018.00240

  11. Rajalakshmi, R., Tiwari, H., Patel, J., Kumar, A., Karthik., R.: Design of kids-specific url classifier using recurrent convolutional neural network. Procedia Comput. Sci. 167, 2124–2131 (2020). https://doi.org/10.1016/j.procs.2020.03.260. https://www.sciencedirect.com/science/article/pii/S1877050920307262, international Conference on Computational Intelligence and Data Science

  12. Rajalakshmi, R., Tiwari, H., Patel, J., Kumar, A., Karthik, R.: Design of kids-specific URL classifier using recurrent convolutional neural network. Procedia Comput. Sci. 167, 2124–2131 (2020)

    Article  Google Scholar 

  13. Rajalakshmi, R., Xaviar, S.: Experimental study of feature weighting techniques for url based webpage classification. Procedia Comput. Sci. 115, 218–225 (2017). https://doi.org/10.1016/j.procs.2017.09.128. https://www.sciencedirect.com/science/article/pii/S1877050917319567. 7th International Conference on Advances in Computing & Communications, ICACC-2017, 22-24 August 2017, Cochin, India

  14. Rajalakshmi, R.: Identifying health domain URLs using SVM. In: Proceedings of the Third International Symposium on Women in Computing and Informatics, pp. 203–208 (2015)

    Google Scholar 

  15. Rajalakshmi, R., Agrawal, R.: Borrowing likeliness ranking based on relevance factor. In: Proceedings of the Fourth ACM IKDD Conferences on Data Sciences (2017)

    Google Scholar 

  16. Rajalakshmi, R., Duraphe, A., Shibani, A.: DLRG@ DravidianLangTech-ACL2022: Abusive comment detection in tamil using multilingual transformer models. In: Proceedings of the Second Workshop on Speech and Language Technologies for Dravidian Languages, pp. 207–213 (2022)

    Google Scholar 

  17. Rajalakshmi, R., More, M., Shrikriti, B., Saharan, G., Samyuktha, H., Nandy, S.: DLRG@TamilNLP-ACL2022: offensive span identification in Tamil using BiLSTM-CRF approach. In: Proceedings of the Second Workshop on Speech and Language Technologies for Dravidian Languages, pp. 248–253. Association for Computational Linguistics, Dublin, Ireland (May 2022). https://doi.org/10.18653/v1/2022.dravidianlangtech-1.38. https://aclanthology.org/2022.dravidianlangtech-1.38

  18. Rajalakshmi, R., Ramraj, S., Ramesh Kannan, R.: Transfer learning approach for identification of malicious domain names. In: International Symposium on Security in Computing and Communication, pp. 656–666. Springer (2018)

    Google Scholar 

  19. Rajalakshmi, R., Reddy, B.Y.: Dlrg@hasoc 2019: an enhanced ensemble classifier for hate and offensive content identification. In: FIRE (2019)

    Google Scholar 

  20. Rajalakshmi, R., Reddy, P., Khare, S., Ganganwar, V.: Sentimental analysis of code-mixed Hindi language. In: Congress on Intelligent Systems, pp. 739–751. Springer (2022). https://doi.org/10.1007/978-981-16-9113-3_54

  21. Rajalakshmi, R., Reddy, Y., Kumar, L.: DLRG@DravidianLangTech-EACL2021: transformer based approach for offensive language identification on code-mixed Tamil. In: Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, pp. 357–362. Association for Computational Linguistics, Kyiv (Apr 2021), https://aclanthology.org/2021.dravidianlangtech-1.53

  22. Rajalakshmi, R., Selvaraj, S., Mattins R., F., Vasudevan, P., Kumar M., A.: Hottest: hate and offensive content identification in Tamil using transformers and enhanced stemming. Comput. Speech Lang. 78, 101464 (2023). https://doi.org/10.1016/j.csl.2022.101464. https://www.sciencedirect.com/science/article/pii/S0885230822000870

  23. Rameshkannan, R., Rajalakshmi, R.: Dlrg@aila 2019: context - aware legal assistance system (2019)

    Google Scholar 

  24. Ravikiran, M., et al.: Findings of the shared task on offensive span identification from Code-mixed Tamil-English comments. In: Proceedings of the Second Workshop on Speech and Language Technologies for Dravidian Languages, pp. 261–270. Association for Computational Linguistics, Dublin, Ireland (May 2022). https://doi.org/10.18653/v1/2022.dravidianlangtech-1.40. https://aclanthology.org/2022.dravidianlangtech-1.40

  25. Samadiani, N., et al.: A review on automatic facial expression recognition systems assisted by multimodal sensor data. Sensors 19(8) (2019). https://doi.org/10.3390/s19081863. https://www.mdpi.com/1424-8220/19/8/1863

  26. Sharen, H., Rajalakshmi, R.: DLRG@ LT-EDI-ACL2022: detecting signs of depression from social media using XGBoost method. In: Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion, pp. 346–349 (2022)

    Google Scholar 

  27. Soubraylu, S., Rajalakshmi, R.: Analysis of sentiment on movie reviews using word embedding self-attentive lstm. Int. J. Ambient Comput. Intell. 12, 33–52 (2021). https://doi.org/10.4018/IJACI.2021040103

  28. Soubraylu, S., Rajalakshmi, R.: Hybrid convolutional bidirectional recurrent neural network based sentiment analysis on movie reviews. Comput. Intell. 37(2), 735–757 (2021). https://doi.org/10.1111/coin.12400. https://onlinelibrary.wiley.com/doi/abs/10.1111/coin.12400

  29. Soubraylu, S., Rajalakshmi, R.: Context-aware sentiment analysis with attention-enhanced features from bidirectional transformers. Soc. Netw. Anal. Min. 12(1), 1–23 (2022). https://doi.org/10.1007/s13278-022-00910-y

    Article  Google Scholar 

  30. Suryawanshi, S., Chakravarthi, B.R., Verma, P., Arcan, M., McCrae, J.P., Buitelaar, P.: A dataset for troll classification of TamilMemes. In: Proceedings of the WILDRE5- 5th Workshop on Indian Language Data: Resources and Evaluation, pp. 7–13. European Language Resources Association (ELRA), Marseille, France (May 2020). https://aclanthology.org/2020.wildre-1.2

Download references

Author information

Authors and Affiliations

  1. Vellore Institute of Technology, Chennai, India

    R. Ramesh Kannan & Ratnavel Rajalakshmi

  2. Georgia Institute of Technology, Atlanta, Georgia

    Manikandan Ravikiran

Authors
  1. R. Ramesh Kannan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manikandan Ravikiran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ratnavel Rajalakshmi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ratnavel Rajalakshmi .

Editor information

Editors and Affiliations

  1. National Institute of Technology Karnataka, Mangalore, India

    Anand Kumar M

  2. National University of Ireland, Galway, Ireland

    Bharathi Raja Chakravarthi

  3. Sri Sivasubramaniya Nadar College of Engineering, Kalavakkam, India

    Bharathi B

  4. National University of Ireland, Galway, Ireland

    Colm O’Riordan

  5. Indian Institute of Technology Madras, Chennai, India

    Hema Murthy

  6. Sri Sivasubramaniya Nadar College of Engineering, Kalavakkam, India

    Thenmozhi Durairaj

  7. University of Hildesheim, Hildesheim, Germany

    Thomas Mandl

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kannan, R.R., Ravikiran, M., Rajalakshmi, R. (2023). MMOD-MEME: A Dataset for Multimodal Face Emotion Recognition on Code-Mixed Tamil Memes. In: M, A.K., et al. Speech and Language Technologies for Low-Resource Languages . SPELLL 2022. Communications in Computer and Information Science, vol 1802. Springer, Cham. https://doi.org/10.1007/978-3-031-33231-9_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-33231-9_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-33230-2

  • Online ISBN: 978-3-031-33231-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation