SpringerLink
Log in

PolitiKweli: A Swahili-English Code-Switched Twitter Political Misinformation Classification Dataset

  • Conference paper
  • First Online:
Speech and Language Technologies for Low-Resource Languages (SPELLL 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2046))

  • 65 Accesses

Abstract

In the age of freedom of speech, users of the various social media platforms post millions of unverified messages, resulting in misinformation. Despite these platforms’ set policies against misinformation, there is an alarming rise in misleading news dissemination. On political matters, misinformation online can result in defamation and in extreme cases, violence offline. Misinformation classification involves classifying text as fake or fact. Most of the existing studies address misinformation classification for posts in a single language only. Among most bilingual or multilingual social media users, code-switching such as Swahili-English, is common practice. This poses a threat to code-switching being used to spread misinformation. There is need for more research in low-resource languages such as Swahili, especially their use to spread misinformation. This study curated the PolitiKweli (dataset: https://github.com/jayneamol/kweli) dataset, a Swahili-English code-switched misinformation classification dataset, containing 6,345 Swahili-English code-switched texts, 22,954 English texts and 211 Swahili texts. The texts are labeled as fake, fact or neutral as compared to fact-checked Twitter dataset also created as part of this study. The paper discusses the dataset curation process including data collection, data processing and data annotation. It also highlights the challenges during the annotation. The study develops a benchmark classification model based on pretrained language model BERT that achieves an F-score of 0.62. The results of the experiment show promising results on the usefulness of the Swahili-English code-switched misinformation classification models. When applied, the classification model can be able to flag instances of misinformation on social media platforms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (Canada)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The aspect of slag is not discussed in this study and left for future work.

  2. 2.

    As part of the larger project, we plan to expand the dataset to include code-switched texts for other local languages.

References

  1. Adelani, D.I., et al.: Masakhaner: named entity recognition for African languages. Trans. Assoc. Comput. Linguist. 9, 1116–1131 (2021)

    Article  Google Scholar 

  2. Ahmed, W., Bath, P.A., Demartini, G.: Using twitter as a data source: an overview of ethical, legal, and methodological challenges. Ethics Online Res. 2, 79–107 (2017)

    Article  Google Scholar 

  3. Altay, S., Berriche, M., Heuer, H., Farkas, J., Rathje, S.: A survey of expert views on misinformation: definitions, determinants, solutions, and future of the field. Harvard Kennedy School Misinformation Rev. 4(4), 1–34 (2023)

    Google Scholar 

  4. Amer, E., Kwak, K.S., El-Sappagh, S.: Context-based fake news detection model relying on deep learning models. Electronics 11(8), 1255 (2022)

    Article  Google Scholar 

  5. Balaton-Chrimes, S.: Who are Kenya’s 42 (+) tribes? The census and the political utility of magical uncertainty. J. Eastern Afr. Stud. 15(1), 43–62 (2021)

    Article  Google Scholar 

  6. Barnett, R., et al.: The lides coding manual: a document for preparing and analyzing language interaction data version 1.1–july 1999. Int. J. Bilingualism 4(2), 131–271 (2000)

    Google Scholar 

  7. Candra, L.K., Qodriani, L.U.: An analysis of code switching in leila s. chudori’s for nadira. Teknosastik 16(1), 9–14 (2019)

    Google Scholar 

  8. Casero-Ripollés, A.: Influencers in the political conversation on twitter: identifying digital authority with big data. Sustainability 13(5), 2851 (2021)

    Article  Google Scholar 

  9. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. ar**v preprint ar**v:1810.04805 (2018)

  10. Guzmán, G.A., Ricard, J., Serigos, J., Bullock, B.E., Toribio, A.J.: Metrics for modeling code-switching across corpora. In: Interspeech, pp. 67–71 (2017)

    Google Scholar 

  11. Hoffmann, C.: In introduction to bilingualism, ed. Logman, Newyork (1991)

    Google Scholar 

  12. Jerit, J., Zhao, Y.: Political misinformation. Annu. Rev. Polit. Sci. 23, 77–94 (2020)

    Article  Google Scholar 

  13. Köchler, H.: Idea and politics of communication in the global age. In: Digital Transformation in Journalism and News Media: Media Management, Media Convergence and Globalization, pp. 7–15 (2017)

    Google Scholar 

  14. Kresse, K., Vierke, C.: Swahili language and literature as resources for Indian ocean studies. Hist. Compass 20(7), e12725 (2022)

    Article  Google Scholar 

  15. Landis, J.R., Koch, G.G.: An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 363–374 (1977)

    Google Scholar 

  16. Magueresse, A., Carles, V., Heetderks, E.: Low-resource languages: a review of past work and future challenges. ar**v preprint ar**v:2006.07264 (2020)

  17. Martin, G., Mswahili, M.E., Jeong, Y.S., Woo, J.: Swahbert: language model of swahili. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 303–313 (2022)

    Google Scholar 

  18. Michail, D., Kanakaris, N., Varlamis, I.: Detection of fake news campaigns using graph convolutional networks. Int. J. Inf. Manag. Data Insights 2(2), 100104 (2022)

    Google Scholar 

  19. Mohammad, S.: A practical guide to sentiment annotation: challenges and solutions. In: Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pp. 174–179 (2016)

    Google Scholar 

  20. Moons, F., Vandervieren, E.: Measuring agreement among several raters classifying subjects into one-or-more (hierarchical) nominal categories. a generalisation of fleiss’ kappa. ar**v preprint ar**v:2303.12502 (2023)

  21. Mozilla: New research: In Kenya, disinformation campaigns seek to discredit pandora papers (2021). https://foundation.mozilla.org/en/blog/new-research-in-kenya-disinformation-campaigns-seek-to-discredit-pandora-papers/

  22. Muhammad, S.H., et al.: Afrisenti: a twitter sentiment analysis benchmark for African languages. ar**v preprint ar**v:2302.08956 (2023)

  23. Mukhongo, L.L.: Participatory media cultures: virality, humour, and online political contestations in Kenya. Afr. Spectr. 55(2), 148–169 (2020)

    Article  Google Scholar 

  24. Nabende, P., Kabiito, D., Babirye, C., Tusiime, H., Nakatumba-Nabende, J.: Misinformation detection in luganda-english code-mixed social media text. ar**v preprint ar**v:2104.00124 (2021)

  25. Ogola, G.: # whatwouldmagufulido? Kenya’s digital “practices” and “individuation” as a (non) political act. J. East. Afr. Stud. 13(1), 124–139 (2019)

    Google Scholar 

  26. Okoth, G.B.W.: How Kenyans on twitter use visuals as a form of political protest. J. Kommunikation. Medien 1–27 (2020)

    Google Scholar 

  27. Ombui, E., Muchemi, L., Wagacha, P.: Hate speech detection in code-switched text messages. In: 2019 3rd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), pp. 1–6. IEEE (2019)

    Google Scholar 

  28. Paullada, A., Raji, I.D., Bender, E.M., Denton, E., Hanna, A.: Data and its (dis) contents: a survey of dataset development and use in machine learning research. Patterns 2(11) (2021)

    Google Scholar 

  29. X developer platform documentation (2023). https://developer.twitter.com/en/docs

  30. Qureshi, M.A., et al.: A novel auto-annotation technique for aspect level sentiment analysis. Comput. Mater. Contin. 70(3), 4987–5004 (2022)

    Google Scholar 

  31. Rajput, K., Kapoor, R., Rai, K., Kaur, P.: Hate me not: detecting hate inducing memes in code switched languages. ar**v preprint ar**v:2204.11356 (2022)

  32. Randolph, J.J.: Free-marginal multirater kappa (multirater k [free]): an alternative to fleiss’ fixed-marginal multirater kappa. Online submission (2005)

    Google Scholar 

  33. Rijhwani, S., Sequiera, R., Choudhury, M., Bali, K., Maddila, C.S.: Estimating code-switching on twitter with a novel generalized word-level language detection technique. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1971–1982 (2017)

    Google Scholar 

  34. Sakketou, F., Plepi, J., Cervero, R., Geiss, H.J., Rosso, P., Flek, L.: Factoid: a new dataset for identifying misinformation spreaders and political bias. ar**v preprint ar**v:2205.06181 (2022)

  35. Shahi, G.K., Dirkson, A., Majchrzak, T.A.: An exploratory study of covid-19 misinformation on twitter. Online Soc. Netw. Media 22, 100104 (2021)

    Article  Google Scholar 

  36. Shu, K., Mahudeswaran, D., Wang, S., Lee, D., Liu, H.: Fakenewsnet: a data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big Data 8(3), 171–188 (2020)

    Article  Google Scholar 

  37. Sitaram, S., Chandu, K.R., Rallabandi, S.K., Black, A.W.: A survey of code-switched speech and language processing. ar**v preprint ar**v:1904.00784 (2019)

  38. Wanjawa, B., Wanzare, L., Indede, F., McOnyango, O., Ombui, E., Muchemi, L.: Kencorpus: a Kenyan language corpus of swahili, dholuo and luhya for natural language processing tasks. ar**v preprint ar**v:2208.12081 (2022)

  39. X: How we address misinformation on x (2023). https://help.twitter.com/en/resources/addressing-misleading-info

  40. Zampieri, M., et al.: Semeval-2020 task 12: multilingual offensive language identification in social media (offenseval 2020). ar**v preprint ar**v:2006.07235 (2020)

Download references

Acknowledgements

We acknowledge the contributions of Shamsuddeen Hassan who assisted with the data collection from Twitter, Martin Okech, Edwin Onkoba and the Maseno University School of Computing and Informatics staff and students. We thank Mary Gitaari, Ezekiel Maina, Nelson Odhiambo, Stephen Otieno, Monicah Odipo, Harrison Kioko, Elphas Otieno, Bowa Marita, Peter Gathuita and Samwel Okonda for their contribution in data annotation.

Author information

Authors and Affiliations

  1. Maseno University, Maseno, Kenya

    Cynthia Amol, Lilian Wanzare & James Obuhuma

Authors
  1. Cynthia Amol
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lilian Wanzare
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James Obuhuma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cynthia Amol .

Editor information

Editors and Affiliations

  1. National University of Ireland, Galway, Ireland

    Bharathi Raja Chakravarthi

  2. Sri Sivasubramaniya Nadar College of Engineering, Kalavakkam, Tamil Nadu, India

    Bharathi B

  3. University of Jaén, Jaén, Jaén, Spain

    Miguel Ángel García Cumbreras

  4. University of Jaén, Jaén, Jaén, Spain

    Salud María Jiménez Zafra

  5. Kongu Engineering College, Erode, Tamil Nadu, India

    Malliga Subramanian

  6. Kongu Engineering College, Erode, Tamil Nadu, India

    Kogilavani Shanmugavadivel

  7. Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, Abu Dhabi, United Arab Emirates

    Preslav Nakov

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Amol, C., Wanzare, L., Obuhuma, J. (2024). PolitiKweli: A Swahili-English Code-Switched Twitter Political Misinformation Classification Dataset. In: Chakravarthi, B.R., et al. Speech and Language Technologies for Low-Resource Languages. SPELLL 2023. Communications in Computer and Information Science, vol 2046. Springer, Cham. https://doi.org/10.1007/978-3-031-58495-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-58495-4_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-58494-7

  • Online ISBN: 978-3-031-58495-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation