SpringerLink
Log in

Explainable Transformer-Based Anomaly Detection for Internet of Things Security

  • Conference paper
  • First Online:
The Seventh International Conference on Safety and Security with IoT (SaSeIoT 2023)

Part of the book series: EAI/Springer Innovations in Communication and Computing ((EAISICC))

Included in the following conference series:

  • 49 Accesses

Abstract

The Internet of Things (IoT) combines sensors and other small devices interconnected locally and via the Internet. Specifically, IoT devices collect data from the environment through sensors, analyze it, and respond to the actual through controllers. The integration of these devices can be seen in various areas like home appliances, healthcare, control systems, etc. On the other hand, massive digital data can drive system performance, and data security is also a serious concern. Therefore, anomaly detection (AD) is necessary to prevent network security infractions and system attacks. Several Artificial Intelligence (AI)-based anomaly detection methods have been designed with higher detection performance; however, they are still “complex” models that are hard to interpret and explain. This chapter proposes a hybrid learning model for AD in IoT with Explainable Artificial Intelligence to enhance the perspective and explainable results. The proposal’s application uses a well-known traffic traces dataset (https://www.kaggle.com/datasets/francoisxa/ds2ostraffictraces). Our code and dataset are added to https://github.com/himanshubeniwal/ml-xai.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (Canada)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 159.99
Price excludes VAT (Canada)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/manujosephv/pytorch_tabular.

References

  1. A. Khanna, S. Kaur, Internet of Things (IoT), applications and challenges: a comprehensive review. Wirel. Personal Commun. 114, 1687–1762 (2020)

    Article  Google Scholar 

  2. S. Chen, X. Hui, D. Liu, H. Bo, H. Wang, A vision of IoT: applications, challenges, and opportunities with China perspective. IEEE Int. Things J. 1(4), 349–359 (2014)

    Article  Google Scholar 

  3. D. Kyriazis, T. Varvarigou, D. White, A. Rossi, J. Cooper, Sustainable smart city IoT applications: Heat and electricity management & eco-conscious cruise control for public transportation, in 2013 IEEE 14th International Symposium on “A World of Wireless, Mobile and Multimedia Networks” (WoWMoM) (IEEE, Piscataway, 2013), pp. 1–5

    Google Scholar 

  4. T. Malche, P. Maheshwarym Internet of things (IoT) for building smart home system, in 2017 International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC) (IEEE, Piscataway, 2017), pp. 65–70

    Google Scholar 

  5. I. Arif, N. Ackovska, IoT aided smart home architecture for anomaly detection, in Data Science and Internet of Things: Research and Applications at the Intersection of DS and IoT (2021), pp. 1–19

    Google Scholar 

  6. M. Min, J.J. Lee, H. Park, K. Lee, Detecting anomalous transactions via an IoT based application: a machine learning approach for horse racing betting. Sensors 21(6), 2039 (2021)

    Google Scholar 

  7. C. Li, L. Mo, H. Tang, R. Yan, Lifelong condition monitoring based on NB-IoT for anomaly detection of machinery equipment. Procedia Manufact. 49, 144–149 (2020)

    Article  Google Scholar 

  8. M. Canizo, I. Triguero, A. Conde, E. Onieva, Multi-head CNN-RNN for multi-time series anomaly detection: an industrial case study. Neurocomputing 363, 246–260 (2019)

    Article  Google Scholar 

  9. W. Di, Z. Jiang, X. **e, X. Wei, Y. Weiren, R. Li, LSTM learning with Bayesian and gaussian processing for anomaly detection in industrial IoT. IEEE Trans. Ind. Inform. 16(8), 5244–5253 (2019)

    Google Scholar 

  10. I.V. Tetko, P. Karpov, R. Van Deursen, G. Godin, State-of-the-art augmented NLP transformer models for direct and single-step retrosynthesis. Nat. Commun. 11(1), 5575 (2020)

    Google Scholar 

  11. L. Sun, C. **a, W. Yin, T. Liang, P.S. Yu, L. He, Mixup-transformer: dynamic data augmentation for NLP tasks (2020). ar**v preprint ar**v:2010.02394

    Google Scholar 

  12. A.A. Cook, G. Mısırlı, Z. Fan, Anomaly detection for IoT time-series data: a survey. IEEE Int. J. 7(7), 6481–6494 (2019)

    Google Scholar 

  13. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  14. D. Liu, G. Liu, A transformer-based variational autoencoder for sentence generation, in 2019 International Joint Conference on Neural Networks (IJCNN) (IEEE, Piscataway, 2019), pp 1–7

    Google Scholar 

  15. X. Qiu, T. Sun, X. Yige, Y. Shao, N. Dai, X. Huang, Pre-trained models for natural language processing: a survey. Sci. China Technol. Sci. 63(10), 1872–1897 (2020)

    Article  Google Scholar 

  16. J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding (2018). ar**v preprint ar**v:1810.04805

    Google Scholar 

  17. S. Karita, N. Chen, T. Hayashi, T. Hori, H. Inaguma, Z. Jiang, M. Someki, N.E.Y. Soplin, R. Yamamoto, X. Wang, et al., A comparative study on transformer vs RNN in speech applications, in 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) (IEEE, Piscataway, 2019), pp. 449–456

    Google Scholar 

  18. H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. **ong, W. Zhang, Informer: Beyond efficient transformer for long sequence time-series forecasting, in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35 (2021), pp. 11106–11115

    Google Scholar 

  19. W. Haixu, X. Jiehui, J. Wang, M. Long, Autoformer: decomposition transformers with auto-correlation for long-term series forecasting. Adv. Neur. Inform. Process. Syst. 34, 22419–22430 (2021)

    Google Scholar 

  20. S. Liu, H. Yu, C. Liao, J. Li, W. Lin, A.X. Liu, S. Dustdar, Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting, in International Conference on Learning Representations (2021)

    Google Scholar 

  21. A. Raza, K.P. Tran, L. Koehl, S. Li, AnoFed: adaptive anomaly detection for digital health using transformer-based federated learning and support vector data description. Eng. Appl. Artif. Intell. 121, 106051 (2023)

    Article  Google Scholar 

  22. A. Raza, K.P. Tran, L. Koehl, S. Li, X. Zeng, K. Benzaidi, Lightweight transformer in federated setting for human activity recognition. ar**v preprint ar**v:2110.00244 (2021)

    Google Scholar 

  23. Z. Chen, D. Chen, X. Zhang, Z. Yuan, X. Cheng, Learning graph structures with transformer for multivariate time-series anomaly detection in IoT. IEEE Int. Things J. 9(12), 9179–9189 (2021)

    Article  Google Scholar 

  24. H. Zhang, Y. **a, T. Yan, G. Liu, Unsupervised anomaly detection in multivariate time series through transformer-based variational autoencoder, in 2021 33rd Chinese Control and Decision Conference (CCDC) (IEEE, Piscataway, 2021), pp. 281–286

    Google Scholar 

  25. S. Tuli, G. Casale, N.R. Jennings, Tranad: Deep transformer networks for anomaly detection in multivariate time series data (2022). ar**v preprint ar**v:2201.07284

    Google Scholar 

  26. F. Zeng, M. Chen, C. Qian, Y. Wang, Y. Zhou, W. Tang, Multivariate time series anomaly detection with adversarial transformer architecture in the internet of things. Future Gener. Comput. Syst. 144, 244–255 (2023)

    Article  Google Scholar 

  27. D. Yadav, X. Zhang, B.T. **, Transformer based anomaly detection on multivariate time series subledger data, in KDD 2023 Workshop on Machine Learning in Finance (MLF) (ACM, New York, NY, USA, 2023). https://www.amazon.science/publications/transformer-based-anomaly-detection-on-multivariate-time-series-subledger-data

    Google Scholar 

  28. A. Alamr, A. Artoli, Unsupervised transformer-based anomaly detection in ECG signals. Algorithms 16(3), 152 (2023)

    Google Scholar 

  29. A. Siva Kumar, S. Raja, N. Pritha, H. Raviraj, R. Babitha Lincy, J. Jency Rubia, An adaptive transformer model for anomaly detection in wireless sensor networks in real-time. Measur. Sensors 25, 100625 (2023)

    Article  Google Scholar 

  30. E. Fernandes, A. Rahmati, K. Eykholt, A. Prakash, Internet of things security research: a rehash of old ideas or new intellectual challenges? IEEE Secur. Privacy 15(4), 79–84 (2017)

    Article  Google Scholar 

  31. S.S. Dhanda, B. Singh, P. **dal, Lightweight cryptography: a solution to secure IoT. Wirel. Personal Commun. 112(3), 1947–1980 (2020)

    Article  Google Scholar 

  32. U. Farooq, N. Tariq, M. Asim, T. Baker, A. Al-Shamma’a, Machine learning and the internet of things security: solutions and open challenges. J. Parall. Distrib. Comput. 162, 89–104 (2022)

    Article  Google Scholar 

  33. N. Tariq, M. Asim, Z. Maamar, M.Z. Farooqi, N. Faci, T. Baker, A mobile code-driven trust mechanism for detecting internal attacks in sensor node-powered IoT. J. Parall. Distrib. Comput. 134, 198–206 (2019)

    Article  Google Scholar 

  34. F. Hussain, R. Hussain, S.A. Hassan, E. Hossain, Machine learning in IoT security: current solutions and future challenges. IEEE Commun. Surv. Tutor. 22(3), 1686–1721 (2020)

    Article  Google Scholar 

  35. B.K. Mohanta, D. Jena, U. Satapathy, S. Patnaik, Survey on IoT security: challenges and solution using machine learning, artificial intelligence and blockchain technology. Int. Things 11, 100227 (2020)

    Article  Google Scholar 

  36. M.S. Mahdavinejad, M. Rezvan, M. Barekatain, P. Adibi, P. Barnaghi, A.P. Sheth, Machine learning for internet of things data analysis: a survey. Digital Commun. Netw. 4(3), 161–175 (2018)

    Article  Google Scholar 

  37. I. Kotenko, I. Saenko, A. Branitskiy, Framework for mobile internet of things security monitoring based on big data processing and machine learning. IEEE Access 6, 72714–72723 (2018)

    Article  Google Scholar 

  38. E. Hossain, I. Khan, F. Un-Noor, S.S. Sikander, M.S.H. Sunny, Application of big data and machine learning in smart grid, and associated security concerns: a review. IEEE Access 7, 13960–13988 (2019)

    Article  Google Scholar 

  39. T. Lin, Y. Wang, X. Liu, X. Qiu, A survey of transformers. AI Open (2022)

    Google Scholar 

  40. K. Choi, C. Hawthorne, I. Simon, M. Dinculescu, J. Engel, Encoding musical style with transformer autoencoders, in International Conference on Machine Learning (PMLR, London, 2020), pp. 1899–1908

    Google Scholar 

  41. S.M. Lundberg, S.-I. Lee, A unified approach to interpreting model predictions, in Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  42. N.X. Hoang, N.V. Hoang, N.H. Du, T.T. Huong, K.P. Tran, et al., Explainable anomaly detection for industrial control system cybersecurity. IFAC-PapersOnLine 55(10), 1183–1188 (2022)

    Article  Google Scholar 

  43. M.T. Ribeiro, S. Singh, C. Guestrin, ”Why should i trust you?” explaining the predictions of any classifier, in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016), pp. 1135–1144

    Google Scholar 

  44. E. Štrumbelj, I. Kononenko, Explaining prediction models and individual predictions with feature contributions. Knowl. Inform. Syst. 41, 647–665 (2014)

    Article  Google Scholar 

  45. S.M. Lundberg, G. Erion, H. Chen, A. DeGrave, J.M. Prutkin, B. Nair, R. Katz, J. Himmelfarb, N. Bansal, S.-I. Lee, From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2(1), 56–67 (2020)

    Article  Google Scholar 

  46. M. Hasan, M.M. Islam, M.I.I. Zarif, M.M.A. Hashem, Attack and anomaly detection in IoT sensors in IoT sites using machine learning approaches. Int. Things 7, 100059 (2019). ISSN 2542-6605. https://doi.org/10.1016/j.iot.2019.100059. https://www.sciencedirect.com/science/article/pii/S2542660519300241

  47. H. Hongsheng, Z. Salcic, L. Sun, G. Dobbie, P.S. Yu, X. Zhang, Membership inference attacks on machine learning: a survey. ACM Comput. Surv. 54(11s), 235 (2022). https://doi.org/10.1145/3523273

  48. N. Rodríguez-Barroso, D. Jiménez-López, M.V. Luzón, F. Herrera, E. Martínez-Cámara, Survey on federated learning threats: concepts, taxonomy on attacks and defences, experimental study and challenges. Inform. Fusion 90, 148–173 (2023). https://doi.org/10.1016/j.inffus.2022.09.011

    Article  Google Scholar 

  49. M. Abadi, A. Chu, I. Goodfellow, H.B. McMahan, I. Mironov, K. Talwar, L. Zhang, Deep learning with differential privacy, in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (ACM, New York, 2016), pp. 308–318. https://doi.org/10.1145/2976749.2978318

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, Mirpur University of Science and Technology (MUST), Mirpur, Pakistan

    A. Saghir

  2. International Chair in DS & XAI, International Research Institute for Artificial Intelligence and Data Science, Dong A University, Danang, Vietnam

    A. Saghir

  3. Indian Institute of Technology Gandhinagar, Gandhinagar, Gujarat, India

    H. Beniwal

  4. International Chair in DS & XAI, International Research Institute for Artificial Intelligence and Data Science, Dong A University, Danang, Vietnam

    H. Beniwal

  5. International Chair in DS & XAI, International Research Institute for Artificial Intelligence and Data Science, Dong A University, Danang, Vietnam

    K. D. Tran

  6. University of Lille, ENSAIT, ULR 2461 - GEMTEX - Génie et Matériaux Textiles, Lille, France

    A. Raza, L. Koehl, X. Zeng & K. P. Tran

  7. International Chair in DS & XAI, International Research Institute for Artificial Intelligence and Data Science, Dong A University, Danang, Vietnam

    A. Raza, L. Koehl, X. Zeng & K. P. Tran

Authors
  1. A. Saghir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. H. Beniwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. K. D. Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. Raza
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. L. Koehl
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. X. Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. K. P. Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. D. Tran .

Editor information

Editors and Affiliations

  1. GEMTEX & ENSAIT Graduate School MADIS-631, University of Lille, ROUBAIX CEDEX 1, France

    Kim Phuc Tran

  2. University of Kent, Canterbury, UK

    Shujun Li

  3. University of Liège, Liège, Belgium

    Cédric Heuchenne

  4. Hanoi University of Science and Technology, Hanoi, Vietnam

    Thu Huong Truong

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Saghir, A. et al. (2024). Explainable Transformer-Based Anomaly Detection for Internet of Things Security. In: Tran, K.P., Li, S., Heuchenne, C., Truong, T.H. (eds) The Seventh International Conference on Safety and Security with IoT. SaSeIoT 2023. EAI/Springer Innovations in Communication and Computing. Springer, Cham. https://doi.org/10.1007/978-3-031-53028-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53028-9_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53027-2

  • Online ISBN: 978-3-031-53028-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation