SpringerLink
Log in

Exploring Dialog Act Recognition in Open Domain Conversational Agents

  • Conference paper
  • First Online:
Big Data Analytics and Knowledge Discovery (DaWaK 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14148))

Included in the following conference series:

  • 501 Accesses

Abstract

Recognizing dialog acts of users is an essential component in building successful conversational agents. In this work, we propose a dialog act (DA) classifier for two of our open domain dialog systems. For this, we first build a hierarchical taxonomy of 8 DAs suitable for classifying user utterances in open-domain setting. Next, we curate a high-quality, multi-domain dataset with over 24k user dialogs and annotate it with our 8 DAs. Next, we fine-tune our pretrained BERT-based DA classifier on this dataset. Through extensive experimentation, we show that our proposed model not only outperforms the baseline SVM classifier by achieving state-of-the-art accuracy but also generalizes extremely well on previously unseen data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Popescu-Belis, A.: Abstracting a dialog act tagset for meeting processing. In: Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004), Lisbon, Portugal. European Language Resources Association (ELRA) (2004). http://www.lrec-conf.org/proceedings/lrec2004/pdf/268.pdf

  2. Wei, C., Yu, Z., Fong, S.: How to build a chatbot: chatbot framework and its capabilities. In: Proceedings of the 2018 10th International Conference on Machine Learning and Computing, pp. 369–373 (2018)

    Google Scholar 

  3. Xu, A., Liu, Z., Guo, Y., Sinha, V., Akkiraju, R.: A new chatbot for customer service on social media. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 3506–3510 (2017)

    Google Scholar 

  4. Malhotra, G., Waheed, A., Srivastava, A., Akhtar, M.S., Chakraborty, T.: Speaker and time-aware joint contextual learning for dialogue-act classification in counselling conversations. In: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 735–745 (2022)

    Google Scholar 

  5. Noble, J.M., et al.: Develo**, implementing, and evaluating an artificial intelligence-guided mental health resource navigation chatbot for health care workers and their families during and following the covid-19 pandemic: Protocol for a cross-sectional study. JMIR Res. Protoc. 11(7), 33717 (2022)

    Article  Google Scholar 

  6. Quinn, K., Zaiane, O.: Identifying questions & requests in conversation. In: Proceedings of the 2014 International C* Conference on Computer Science & Software Engineering, pp. 1–6 (2014)

    Google Scholar 

  7. Welivita, A., Pu, P.: A taxonomy of empathetic response intents in human social conversations. In: Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain, pp. 4886–4899. International Committee on Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.coling-main.429. https://aclanthology.org/2020.coling-main.429

  8. Godfrey, J.J., Holliman, E.: Switchboard-1 release 2. Linguistic Data Consortium, Philadelphia, vol. 926, p. 927 (1997)

    Google Scholar 

  9. Dhillon, R., Bhagat, S., Carvey, H., Shriberg, E.: Meeting recorder project: dialog act labeling guide. Technical report, International Computer Science Inst Berkeley CA (2004)

    Google Scholar 

  10. Colombo, P., Chapuis, E., Manica, M., Vignon, E., Varni, G., Clavel, C.: Guiding attention in sequence-to-sequence models for dialogue act prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 7594–7601 (2020)

    Google Scholar 

  11. Li, R., Lin, C., Collinson, M., Li, X., Chen, G.: A dual-attention hierarchical recurrent neural network for dialogue act classification. In: Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pp. 383–392 (2019)

    Google Scholar 

  12. Raheja, V., Tetreault, J.: Dialogue act classification with context-aware self-attention. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 3727–3733 (2019)

    Google Scholar 

  13. Qin, L., Che, W., Li, Y., Ni, M., Liu, T.: DCR-Net: a deep co-interactive relation network for joint dialog act recognition and sentiment classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 8665–8672 (2020)

    Google Scholar 

  14. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171–4186. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/N19-1423. https://aclanthology.org/N19-1423

  15. Saha, T., Gupta, D., Saha, S., Bhattacharyya, P.: Emotion aided dialogue act classification for task-independent conversations in a multi-modal framework. Cogn. Comput. 1–13 (2020)

    Google Scholar 

  16. Gautam, D., Maharjan, N., Graesser, A.C., Rus, V.: Automated speech act categorization of chat utterances in virtual internships. In: EDM (2018)

    Google Scholar 

  17. Zhang, R., Gao, D., Li, W.: What are tweeters doing: Recognizing speech acts in twitter. In: Analyzing Microtext (2011)

    Google Scholar 

  18. Yu, D., Yu, Z.: Midas: A dialog act annotation scheme for open domain humanmachine spoken conversations. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 1103–1120 (2021)

    Google Scholar 

  19. Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., Toutanova, K.: Boolq: exploring the surprising difficulty of natural yes/no questions. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 2924–2936 (2019)

    Google Scholar 

  20. Coucke, A., et al.: Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. ar**v preprint ar**v:1805.10190 (2018)

  21. Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad: 100,000+ questions for machine comprehension of text. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383–2392 (2016)

    Google Scholar 

  22. Byrne, B., et al.: Taskmaster-1: toward a realistic and diverse dialog dataset. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 4516–4525 (2019)

    Google Scholar 

  23. Hemphill, C.T., Godfrey, J.J., Doddington, G.R.: The ATIS spoken language systems pilot corpus. In: Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, 24–27 June 1990 (1990)

    Google Scholar 

  24. Acharya, S., Fung, G.: Using optimal embeddings to learn new intents with few examples: an application in the insurance domain (2020)

    Google Scholar 

  25. González-Carvajal, S., Garrido-Merchán, E.C.: Comparing bert against traditional machine learning text classification. ar**v preprint ar**v:2005.13012 (2020)

  26. Joachims, T.: Text categorization with support vector machines: learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 137–142. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0026683

    Chapter  Google Scholar 

  27. Luo, X.: Efficient English text classification using selected machine learning techniques. Alex. Eng. J. 60(3), 3401–3409 (2021). https://doi.org/10.1016/j.aej.2021.02.009

    Article  Google Scholar 

  28. Morales-Hernández, R.C., Becerra-Alonso, D., Vivas, E.R., Gutiérrez, J.: Comparison between SVM and distilbert for multi-label text classification of scientific papers aligned with sustainable development goals. In: Pichardo Lagunas, O., Martínez-Miranda, J., Martínez Seis, B. (eds.) MICAI 2022. LNCS, vol. 13613, pp. 57–67. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19496-2_5

  29. Kambar, M.E.Z.N., Nahed, P., Cacho, J.R.F., Lee, G., Cummings, J., Taghva, K.: Clinical text classification of Alzheimer’s drugs’ mechanism of action. In: Yang, X.-S., Sherratt, S., Dey, N., Joshi, A. (eds.) Proceedings of Sixth International Congress on Information and Communication Technology. LNNS, vol. 235, pp. 513–521. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-2377-6_48

    Chapter  Google Scholar 

  30. Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  31. Chen, Y., Liu, Y., Chen, L., Zhang, Y.: DialogSum: a real-life scenario dialogue summarization dataset. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 5062–5074. Association for Computational Linguistics, Online (2021). https://doi.org/10.18653/v1/2021.findings-acl.449. https://aclanthology.org/2021.findings-acl.449

  32. Li, Y., Su, H., Shen, X., Li, W., Cao, Z., Niu, S.: DailyDialog: a manually labelled multi-turn dialogue dataset. In: Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Taipei, Taiwan, pp. 986–995. Asian Federation of Natural Language Processing (2017). https://aclanthology.org/I17-1099

  33. Sun, K., Yu, D., Chen, J., Yu, D., Choi, Y., Cardie, C.: DREAM: a challenge data set and models for dialogue-based reading comprehension. Trans. Assoc. Comput. Linguist. 7, 217–231 (2019). https://doi.org/10.1162/tacl_a_00264

    Article  Google Scholar 

  34. Cui, L., Wu, Y., Liu, S., Zhang, Y., Zhou, M.: MuTual: a dataset for multi-turn dialogue reasoning. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1406–1416. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.130. https://aclanthology.org/2020.acl-main.130

Download references

Author information

Authors and Affiliations

  1. Computing Science, University of Alberta, Edmonton, AB, Canada

    Maliha Sultana & Osmar R. Zaíane

Authors
  1. Maliha Sultana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Osmar R. Zaíane
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maliha Sultana .

Editor information

Editors and Affiliations

  1. Poznań University of Technology, Poznan, Poland

    Robert Wrembel

  2. Free University of Bozen-Bolzano, Bozen-Bolzano, Italy

    Johann Gamper

  3. Johannes Kepler University Linz, Linz, Austria

    Gabriele Kotsis

  4. Vienna University of Technology, Vienna, Austria

    A Min Tjoa

  5. Johannes Kepler University Linz, Linz, Austria

    Ismail Khalil

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sultana, M., Zaíane, O.R. (2023). Exploring Dialog Act Recognition in Open Domain Conversational Agents. In: Wrembel, R., Gamper, J., Kotsis, G., Tjoa, A.M., Khalil, I. (eds) Big Data Analytics and Knowledge Discovery. DaWaK 2023. Lecture Notes in Computer Science, vol 14148. Springer, Cham. https://doi.org/10.1007/978-3-031-39831-5_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-39831-5_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-39830-8

  • Online ISBN: 978-3-031-39831-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation