SpringerLink
Log in

Evolving Cybersecurity Challenges in the Age of AI-Powered Chatbots: A Comprehensive Review

  • Conference paper
  • First Online:
Technological Innovation for Human-Centric Systems (DoCEIS 2024)

Part of the book series: IFIP Advances in Information and Communication Technology ((IFIPAICT,volume 716))

Included in the following conference series:

  • 96 Accesses

Abstract

In today’s world of super-digitization and dynamic transformation, the Artificial Intelligence (AI)-based chatbot is a revolutionary stride in technology. This innovative chatbot type is characterized by unmatched technological advancement across the globe in terms of efficiency and interactivity within several fields. Moving from simple automated scripts to sophisticated natural language processing systems like Chat GPT stands as a remarkable leap in conversational technology. However, this development conveys distinct computer security threats, as AI upholds a dual role in strengthening and weakening digital security. This study explores the conceivable risks that come with AI-Chatbots, as well as the appropriate mitigation strategies. This paper outlines the current risks, threats, and consequences posed by these digital assistants, and discusses useful strategies and methods to mitigate these risks and protect personal data and sensitive information. Combining current research and perspectives, to provide a balanced view of the challenges and opportunities that AI-Chatbots present in the cybersecurity domain. Establishing a guide to the development of my PhD proposal, an Intelligent Chatbot that combines the most advanced LLMs with its databases, in a safe, functional, and user-friendly user interface.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Gianvecchio, S., **e, M., Wu, Z., Wang, H.: Humans and bots in internet chat: measurement analysis, and automated classification. IEEEACM Trans. Netw. 19, 1557–1571 (2011)

    Google Scholar 

  2. Shawar, B.A., Atwell, E.: Chatbots: Are they really useful? J. Lang. Technol. Comput. Linguist. 22, 29–49 (2007)

    Google Scholar 

  3. Gupta, A., Hathwar, D., Vijayakumar, A.: Introduction to AI chatbots. Int. J. Eng. Res. Technol. 9, 07 (2020)

    Google Scholar 

  4. Tavares, A., Sousa, P., Proença, R.: Exploring the dynamics between artificial intelligence and cybersecurity in Healthcare. ARIS2 - Adv. Res. Inf. Syst. Secur. 4(1), 20–34 (2024). https://doi.org/10.56394/aris2.v4i1.44

  5. Yang, J., Chen, Y.-L., Por, L.Y., Ku, C.S.: A systematic literature review of information security in chatbots. Appl. Sci. 13, 6355 (2023)

    Google Scholar 

  6. Iqbal, F., Samsom, F., Kamoun, F., MacDermott, Á.: When chatgpt goes rogue: exploring the potential cybersecurity threats of AI-powered conversational chatbots. Front. Commun. Netw. 4, 1220243 (2023)

    Google Scholar 

  7. Mateus-Coelho, N.: Cybersecurity in our world (2022)

    Google Scholar 

  8. Sule, D.: Social engineering as a tool for warfare: a look at the 2022 dollar bill rejection hoax in Nigeria. ARIS2 - Adv. Res. Inf. Syst. Secur. 3(2), 3–10 (2023). https://doi.org/10.56394/aris2.v3i2.29

  9. Soni, B., Gautam, A., Soni, G.: Exploring the advancements and implications of artificial intelligence. Int. J. Sci. Res. Eng. Manag. (2023)

    Google Scholar 

  10. Kooli, C.: Ethical implications of AI chatbots in education and research. Sustainability 15(7), 5614 (2023). https://doi.org/10.3390/su15075614

  11. Gupta, M., Akiri, C., Aryal, K., Parker, E., Praharaj, L.: From ChatGpt to ThreatGpt: Impact of generative AI in cybersecurity and privacy. IEEE Access 11 (2023). https://doi.org/10.1109/ACCESS.2023.3300381

  12. Ferreira, J., Mamede, H.S., Mateus-Coelho, N.: Risk management in the current digital reality of organizations. In: Contemporary Challenges for Cyber Security and Data Privacy, IGI Global, pp. 31–50 (2023)

    Google Scholar 

  13. Sarker, H., Janicke, H., Mohammad, N., Watters, P., Nepal, S.: Human-AI teaming in cybersecurity. Ar**v Prepr. ar**v:2310.12162 (2023)

  14. Bullee, W., Junger, M.: Social Engineering (2020)

    Google Scholar 

  15. Duarte, N., Coelho, N., Guarda, T.: Social engineering: The art of attacks. Algoritmi Center Minho Univercity (2023)

    Google Scholar 

  16. Usmani, U.A., Happonen, A., Watada, J.: Human-Centered Artificial intelligence: designing for user empowerment and ethical considerations. In: IEEE Humanitarian Opportunities and Risk Assessment (HORA) 2023. https://doi.org/10.1109/HORA58378.2023.10156761

  17. Amershi, S., et al.: Guidelines for human-AI interaction. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, New York, NY, USA: Association for Computing Machinery, pp. 1–13 (2019). https://doi.org/10.1145/3290605.3300233

  18. Costa, A., Silva F.: Interaction design for AI systems: an oriented state-of-the-art. In: 2022 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), pp. 1–7 (2022). https://doi.org/10.1109/HORA55278.2022.9800084

  19. Zeng, Y.: How human centered AI will contribute towards intelligent gaming systems. AAAI Conf. Artif. Intell. 35(18), 17868 (2021). https://doi.org/10.1609/aaai.v35i18.17868

  20. Battistoni, P., Gregorio, M.D., Romano, M., Sebillo, M., Vitiello, G.: Can AI-oriented requirements enhance human-centered design of intelligent interactive systems? results from a workshop with young HCI designers. Multimodal Technol. Interact. 7(3), 24 (2023). https://doi.org/10.3390/mti7030024

  21. Böhm, S., et al.: Intent identification and analysis for user-centered Chatbot design: a case study on the example of recruiting Chatbots in Germany. Publ. (2022)

    Google Scholar 

  22. Page, M., et al.: The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. J. Clin. Epidemiol. (2021). https://doi.org/10.1016/j.jclinepi.2021.03.001

  23. Weizenbaum, J.: Eliza - a computer program for the study of natural language communication between man and machine. Commun. ACM 9(1), 36–45 (1966)

    Google Scholar 

  24. Wallace, R.S.: The anatomy of A.l.I.C.E (2004)

    Google Scholar 

  25. Shawar, B.A., Atwell, E.: Using corpora in machine-learning chatbot systems. Int. J. Corpus Linguist. 10(4), 489–516 (2005)

    Google Scholar 

  26. Lowe, R., Noseworthy, M., Serban, I.V., Angelard-Gontier, N., Bengio, Y., Pineau, J.: Towards an automatic turing test: learning to evaluate dialogue responses. In: Proceedings of the 5th International Conference on Learning Representations (ICLR) Workshop, Toulon, France (2017)

    Google Scholar 

  27. Liddy, D.: Natural Language Processing: Encyclopedia of Library and Information Science, 2nd edn. Marcel Decker Inc, NY (2001)

    Google Scholar 

  28. Research, V.: Chatbot market size and share analysis, Ind. Rep. 2014–2025 (2018)

    Google Scholar 

  29. Vaswani, A., et al.: Attention is all you need. In: Advances in neural information processing systems, Curran Associates, Inc. (2017)

    Google Scholar 

  30. Syed, A.M., Zafar, H., Malik, N.: Social engineering: Concepts, techniques, and security countermeasures. J. Comput. Virol. Hacking Tech. 16(4), 265–278 (2020)

    Google Scholar 

  31. Mann, I.: Hacking the Human: Social Engineering Techniques and Security Counter measures. Gower Publishing (2008)

    Google Scholar 

  32. Ferreira, D.J., Mateus-Coelho, N., Mamede, H.S.: Methodology for predictive cyber security risk assessment (pcsra). Procedia Comput. Sci. 219, 1555–1563 (2023)

    Google Scholar 

  33. Sebastian, G.: Do chatgpt and other AI Chatbots pose a cybersecurity risk? An exploratory study. Int. J. Secur. Priv. Pervasive Comput. 15 (2023)

    Google Scholar 

  34. McIntire, J.P., McIntire, L.K., Havig, P.R.: Methods for Chatbot detection in distributed textbased communications. In: 2010 International Symposium on Collaborative Technologies and Systems, pp. 463–472 (2010)

    Google Scholar 

  35. Siddiqi, A., Pak, W., Siddiqi, M.A.: Social engineering based cyberattacks and countermeasures. Appl. Sci. 12(12), 6042 (2022)

    Google Scholar 

  36. Chinmulgund, A., Khatwani, R., Tapas, P., Shah, P., Sekhar, R.: Anthropomorphism of AI based Chatbots by users during communication. In: Proceedings of the IEEE Conference on Information Technology (2023)

    Google Scholar 

  37. Biswas, D.: Privacy preserving Chatbot conversations. In: 2020 IEEE Third International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), pp. 179–182 (2020)

    Google Scholar 

  38. Bozic, J., Wotawa, F.: Security testing for chatbots. In: Medina-Bulo, I., Merayo, M.G., Hierons, R. (eds.) ICTSS 2018. LNCS, vol. 11146, pp. 33–38. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99927-2_3

    Chapter  Google Scholar 

  39. Cheng, Y., Jiang, H.: How do AI-driven Chatbots impact user experience? Examining gratifications, perceived privacy risk, satisfaction, loyalty, and continued use. J. Broadcast. Electron. Media, 64, 592–614 (2020)

    Google Scholar 

  40. Ischen, C., Araujo, T., Voorveld, H., van Noort, G., Smit, E.: Privacy concerns in Chatbot interactions. 34–48 (2019). https://doi.org/10.1007/978-3-030-39540-7_3

  41. Hasal, M., Nowaková, J., Ahmed Saghair, K., Abdulla, H., Snášel, V., Ogiela, L.: Chatbots: security, privacy, data protection, and social aspects. Concurr. Comput. Pract. Exp. 33, e6426 (2021)

    Google Scholar 

  42. Al-Saadoon, M.W., Al-Bayatti, H.M.: A comparison of Trojan virus behavior in Linux and Windows operating systems. Ar**v Prepr. abs11051234 (2011)

    Google Scholar 

  43. Smys, S., Wang, H.: Naive Bayes and entropy based analysis and classification of humans and chat bots. 3, 40–49 (2021)

    Google Scholar 

  44. Bailey, N.R., Scerbo, M.W.: Automation-induced complacency for monitoring highly reliable systems: the role of task complexity, system experience, and operator trust. Theor. Issues Ergon. Sci. 8, 321–348 (2007)

    Google Scholar 

  45. Parasuraman, R., Manzey, D.: Complacency and bias in human use of automation: an attentional integration. Hum. Factors J. Hum. Factors Ergon. Soc. 52, 381–410 (2010)

    Google Scholar 

  46. Singh, I.L., Molloy, R., Parasuraman, R.: Automation-induced “complacency”: development of the complacency potential rating scale. Int. J. Aviat. Psychol. 3, 111–122 (1993)

    Google Scholar 

  47. Manzey, D., Bahner, J.E., Hueper, A.D.: Misuse of automated aids in process control: complacency, automation bias and possible training interventions. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 220–224 (2006)

    Google Scholar 

  48. Rodriguez, S.S., O’Donovan, J., Schaffer, J., Höllerer, T. Knowledge complacency and decision support systems. In: 2019 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA), pp. 43–51 (2019)

    Google Scholar 

  49. **, S.V., Youn, S.: Social presence and imagery processing as predictors of Chatbot continuance intention in human-AI-interaction. Int. J. Hum.-Comput. Interact. (2022)

    Google Scholar 

  50. Ribeiro, R.A.B.: Improving social engineering resilience in enterprises. Master’s Dissertation in Information and Enterprise Systems (MISE), Universidade Aberta (2023)

    Google Scholar 

  51. Zimmermann, G.T., Nagappan, N.: The state of the ML universe: 10 years of artificial intelligence & machine learning software development on GitHub. In: Proceedings of the 17th International Conference on Mining Software Repositories (2020)

    Google Scholar 

  52. von Ahn, M., Blum, N., Hopper, Langford, J.: Captcha: using hard AI problems for security. Proc. Eurocrypt. 294–311 (2003)

    Google Scholar 

  53. Mohta, A.: Bots are back in yahoo chat rooms (2007). http://www.technospot.net/blogs/bots-are-back-in-yahoo-chat-room/

  54. Mohta: Yahoo chat: captcha check to remove bots (2007). http://www.technospot.net/blogs/yahoo-chat-captcha-check-to-remove-bots/

  55. Malatji: Systematic literature review on AI-driven cyberattacks and countermeasures. In: Proceedings of the 2023 International Conference on Data and Applications Technology (2023)

    Google Scholar 

  56. Chanda, R., Prabhu, S.: Secured framework for banking Chatbots using AI, ML and NLP. In 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS), pp. 60–65 (2023)

    Google Scholar 

  57. Bresniker, M., Gavrilovska, A., Holt, J., Milojicic, D., Tran, T.-D.: Grand challenge: applying artificial intelligence and machine learning to cybersecurity. Computer 52, 45–52 (2019)

    Google Scholar 

  58. Lai, S.-T., Leu, F.-Y., Lin, J.-W.: A banking Chatbot security control procedure for protecting user data security and privacy. 561–571 (2018)

    Google Scholar 

  59. Zeadally, S., Adi, E., Baig, Z., Khan, A.: Harnessing artificial intelligence capabilities to improve cybersecurity. IEEE Access 8, 23817–23837 (2020)

    Google Scholar 

  60. Ansari, M., Sharma, P.K., Dash, B.: Prevention of phishing attacks using AI-based cybersecurity awareness training. Int. J. Smart Sens. Adhoc Netw. (2022)

    Google Scholar 

  61. Jobin, A., Ienca, M., Vayena, E.: Artificial intelligence: the global landscape of ethics guidelines. ETH Zurich (2019)

    Google Scholar 

  62. Shneiderman, B.: Human-centered artificial intelligence: reliable, safe & trustworthy. Int. J. Hum.–Comput. Interact. 36(6), 495–504 (2020). https://doi.org/10.1080/10447318.2020.1741118

Download references

Author information

Authors and Affiliations

  1. LAPI2S - Laboratory of Privacy and Information, Lusófona University, Lisbon, Portugal

    André F. Costa

  2. CTS - Center of Technology and Systems, Lusófona University, Lisbon, Portugal

    Nuno Mateus Coelho

Authors
  1. André F. Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nuno Mateus Coelho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to André F. Costa .

Editor information

Editors and Affiliations

  1. NOVA University Lisbon, Monte da Caparica, Portugal

    Luis M. Camarinha-Matos

  2. NOVA University Lisbon, Monte da Caparica, Portugal

    Filipa Ferrada

Rights and permissions

Reprints and permissions

Copyright information

© 2024 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Costa, A.F., Coelho, N.M. (2024). Evolving Cybersecurity Challenges in the Age of AI-Powered Chatbots: A Comprehensive Review. In: Camarinha-Matos, L.M., Ferrada, F. (eds) Technological Innovation for Human-Centric Systems. DoCEIS 2024. IFIP Advances in Information and Communication Technology, vol 716. Springer, Cham. https://doi.org/10.1007/978-3-031-63851-0_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-63851-0_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-63850-3

  • Online ISBN: 978-3-031-63851-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation