Abstract
Online discussion forums serve as dynamic environments where students and teachers collaboratively generate and utilize a wealth of content for knowledge sharing and assessment. The research involved 18 informatics graduates at the Faculty of Science at the University of Mostar, Bosnia and Herzegovina and two teachers who extracted and analyzed transcripts from an online forum, which was part of the online course “Evaluation of E-Learning Systems (EES)" held on Moodle during the winter semester of the academic year 2022/2023. The paper introduces a novel Natural Language Processing (NLP) approach to evaluating student contributions by contrasting their postings with corresponding instructional materials. Utilizing text similarity measurement, the research addresses key questions: Does the content extracted from individual student postings reflect student knowledge on a given topic? Do similarity scores align with human rankings of contribution relevance? Do students equally benefit from collaborative learning? The research evaluates the efficacy of five multilingual sentence embedding models and integrates human analysis to assess the relevance of students’ contributions. Contributions of this study include the evaluation of multilingual sentence embedding models and a thorough examination of the human-perceived relevance of student contributions. The findings aim to enhance the understanding of whether this approach can effectively assess and validate the educational value of student discussions within online forums and contribute to the optimization of collaborative learning experiences.
The presented results are the outcome of the research project “Enhancing Adaptive Courseware based on Natural Language Processing” undertaken with the support of the United States Office of Naval Research Grant (N00014-20-1-2066).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
S. Stankov, D. Vasić: Evaluation of the e-learning system (October 2022 - designed for students enrolled in the EES 22/23 course), Mostar, 2022.
References
Koloski, B., Pollak, S., Škrlj, B., Martinc, M.: Extending neural keyword extraction with TF-IDF tagset matching. In: Proceedings of the EACL Hackashop on News Media Content Analysis and Automated Report Generation, Kiev, Ukraine, pp. 22–29 (2021)
Singh, A., Deepak, P., Raghu, D.: Retrieving similar discussion forum threads: a structure based approach. In: Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (2012)
Shaik, T., et al.: A review of the trends and challenges in adopting natural language processing methods for education feedback analysis, ar**v:2301.08826v1 [cs.CL] (2023)
Hamadi, H., Tafili, A., Kates, F.R.: Exploring an innovative approach to enhance discussion board engagement. TechTrends 67, 741–751 (2023). https://doi.org/10.1007/s11528-023-00850-0
Azevedo, B.F.T., Behar, P.A., Reategui, E.: Qualitative Analysis of Discussion Forums (2011)
Pinheiro, A., Ferreira, R., Ferreira, M.A.D., Rolim, V.B., Tenório, J.V.S.: Statistical and semantic features to measure sentence similarity in Portuguese. In: Proceedings 2017 Brazilian Conference on Intelligent Systems (BRACIS), Uberlandia, Brazil, pp. 342–347 (2017). https://doi.org/10.1109/BRACIS.2017.40
Sun, X., et al.: Sentence similarity based on contexts. Trans. Assoc. Comput. Linguist. 10, 573–588 (2022). https://doi.org/10.1162/tacl_a_00477
De Lima, D.P., Gerosa, M.A., Conte, T.U., et al.: What to expect, and how to improve online discussion forums: the instructors’ perspective. J. Internet Serv. Appl. 10, 1–15 (2019). https://doi.org/10.1186/s13174-019-0120-0
Banawan, M.P., Shin, J., Arner, T., Balyan, R., Leite, W.L., McNamara, D.S.: Shared language: linguistic similarity in an algebra discussion forum. Computers 12(3), 53 (2023). https://doi.org/10.3390/computers12030053
Zarra, T., Raddouane, C., Rdouan, F., El Afia, A.: Using textual similarity and sentiment analysis in discussions forums to enhance learning. Int. J. Softw. Eng. Appl. 10, 191–200 (2016)
Reimers, N., Gurevych, I.: Making monolingual sentence embeddings multilingual using knowledge distillation. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (2020)
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (2019)
Weikang, W., Guanhua, C., Hanqing, W., Yue, H., Yun, C.: Multilingual sentence transformer as a multilingual word aligner ar**v (2023)
Conneau, A., et al.: Unsupervised cross-lingual representation learning at scale. In: Proceedings of the 58th Conference of the Association for Computational Linguistics, ACL 2020, Virtual Conference, July 6-8, 2020, pp. 8440–8451 (2020)
Lauscher, L., Ravishankar, V., Vulic, I., Glavaš, G.: From zero to hero: on the limitations of zero-shot language transfer with multilingual Transformers. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4483–4499, Online (2020b)
Artetxe, M., Ruder, S., Yogatama, D.: On the cross-lingual transferability of monolingual representations. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4623–4637, Online. Association for Computational Linguistics (2020)
Abadaoui, A., Dutta, S.: Attention over pre-trained Sentence Embeddings for Long Document Classification, ReNeuIR 2023. In: Workshop on Reaching Efficiency in Neural Information Retrieval, ar**v:2307.09084v1 [cs.CL] (2023)
Vasić, D., Brajković, E.: Development and evaluation of word embeddings for morphologically rich languages. In: Proceedings 2018 26th International Conference on Software, Telecommunications and Computer Networks (SoftCOM), Split, Croatia, pp. 1–5 (2018). https://doi.org/10.23919/SOFTCOM.2018.8555822
**aofei, S., et al.: Sentence similarity based on contexts. Trans. Assoc. Comput. Linguist. 10, 573–588 (2022). https://doi.org/10.1162/tacl_a_00477
Kuzilek, J., Zdrahal, Z., Vaclavek, J., Fuglik, V., Skocilas, J.: Exploring exam strategies of successful first year engineering students. In: Proceedings of the Tenth International Conference on Learning Analytics & Knowledge (LAK 2020), Frankfurt, Germany, pp. 124-128. Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3375462.3375469
Acknowledgments
The presented results are the outcome of the research project “Enhancing Adaptive Courseware based on Natural Language Processing” undertaken with the support of the United States Office of Naval Research Grant (N00014-20-1-2066).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Vasić, D., Stankov, S., Gašpar, A. (2024). Enhancing Student Discussion Forum Analysis Through Natural Language Processing. In: Volarić, T., Crnokić, B., Vasić, D. (eds) Digital Transformation in Education and Artificial Intelligence Application. MoStart 2024. Communications in Computer and Information Science, vol 2124. Springer, Cham. https://doi.org/10.1007/978-3-031-62058-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-62058-4_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-62057-7
Online ISBN: 978-3-031-62058-4
eBook Packages: Computer ScienceComputer Science (R0)