SpringerLink
Log in

Enhancing the Parallel UC2B Framework: Approach Validation and Scalability Study

  • Conference paper
  • First Online:
Computational Science – ICCS 2024 (ICCS 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14838))

Included in the following conference series:

  • 94 Accesses

Abstract

Anomaly detection is a critical aspect of uncovering unusual patterns in data analysis. This involves distinguishing between normal patterns and abnormal ones, which inherently involves uncertainty. This paper presents an enhanced version of the parallel UC2B framework for anomaly detection, previously introduced in a different context. In this work, we present an extension of the framework and present its large-scale evaluation on the Supercomputer Fugaku. The focus is on assessing its scalability by leveraging a great number of nodes to process large-scale datasets within the cybersecurity domain, using the UNSW-NB15 dataset. The ensemble learning techniques and inherent parallelizability of the Unite and Conquer approach are highlighted as key components, contributing to the framework’s computational efficiency, scalability, and accuracy. This study expands upon the framework’s capabilities and emphasizes its potential integration into an existing Security Orchestration, Automation, and Response (SOAR) system for enhancing cyber threat detection and response.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Thailand)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 128.39
Price includes VAT (Thailand)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 73.99
Price excludes VAT (Thailand)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Supercomputer Fugaku - Supercomputer Fugaku, A64FX 48C 2.2 GHz, Tofu Interconnect D. https://www.top500.org/system/179807/

  2. Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: GANomaly: semi-supervised anomaly detection via adversarial training. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds.) Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, 2–6 December 2018, Revised Selected Papers, Part III 14. LNCS, Vol. 11363, pp. 622–637. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20893-6_39

  3. An, J., Cho, S.: Variational autoencoder based anomaly detection using reconstruction probability. Special Lect. IE 2(1), 1–18 (2015)

    Google Scholar 

  4. Anandakrishnan, A., Kumar, S., Statnikov, A., Faruquie, T., Xu, D.: Anomaly detection in finance: editors’ introduction. In: KDD 2017 Workshop on Anomaly Detection in Finance, pp. 1–7. PMLR (2018)

    Google Scholar 

  5. Bukhari, O., Agarwal, P., Koundal, D., Zafar, S.: Anomaly detection using ensemble techniques for boosting the security of intrusion detection system. Procedia Comput. Sci. 218, 1003–1013 (2023). https://doi.org/10.1016/j.procs.2023.01.080

  6. Cappello, F., Geist, A., Gropp, W., Kale, S., Kramer, B., Snir, M.: Toward exascale resilience: 2014 update. Supercomputing Front. Innov. 1(1), 5–28 (2014). https://doi.org/10.14529/jsfi140101, https://superfri.org/index.php/superfri/article/view/14

  7. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 1–58 (2009). https://doi.org/10.1145/1541880.1541882

  8. Diop, A., Emad, N., Winter, T.: A parallel and scalable framework for insider threat detection. In: 27th IEEE International Conference on High Performance Computing, Data, and Analytics, HiPC 2020, Pune, India, 16–19 December 2020, pp. 101–110. IEEE (2020)

    Google Scholar 

  9. Diop, A., Emad, N., Winter, T.: A unite and conquer based ensemble learning method for user behavior modeling. In: 39th IEEE International Performance Computing and Communications Conference, IPCCC 2020, Austin, TX, USA, 6–8 November 2020, pp. 1–8. IEEE (2020)

    Google Scholar 

  10. Du, Q., Tang, B., **e, W., Li, W.: Parallel and distributed computing for anomaly detection from hyperspectral remote sensing imagery. Proc. IEEE 109(8), 1306–1319 (2021). https://doi.org/10.1109/JPROC.2021.3076455

    Article  Google Scholar 

  11. Emad, N., Petiton, S.G.: Unite and conquer approach for high scale numerical computing. J. Comput. Sci. 14, 5–14 (2016)

    Article  MathSciNet  Google Scholar 

  12. Ghiasvand, S., Ciorba, F.M.: Anomaly detection in high performance computers: a vicinity perspective. In: 2019 18th International Symposium on Parallel and Distributed Computing (ISPDC), pp. 112–120 (2019). https://doi.org/10.1109/ISPDC.2019.00024

  13. Görnitz, N., Braun, M., Kloft, M.: Hidden Markov anomaly detection. In: International Conference on Machine Learning, pp. 1833–1842. PMLR (2015)

    Google Scholar 

  14. Humble, R., et al.: Beam-based RF station fault identification at the SLAC Linac coherent light source. Phys. Rev. Accel. Beams 25, 122804 (2022). https://doi.org/10.1103/PhysRevAccelBeams.25.122804

  15. Humble, R., Zhang, Z., O’Shea, F., Darve, E., Ratner, D.: Coincident learning for unsupervised anomaly detection (2023)

    Google Scholar 

  16. Komolafe, T., Quevedo, A.V., Sengupta, S., Woodall, W.H.: Statistical evaluation of spectral methods for anomaly detection in static networks. Netw. Sci. 7(2), 238–267 (2019)

    Google Scholar 

  17. Malhotra, P., Vig, L., Shroff, G., Agarwal, P., et al.: Long short term memory networks for anomaly detection in time series. In: ESANN, vol. 2015, p. 89 (2015)

    Google Scholar 

  18. Moustafa, N., Slay, J.: The evaluation of network anomaly detection systems: statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set, pp. 1–14, January 2016. https://doi.org/10.1080/19393555.2015.1125974

  19. Nakao, M., Ueno, K., Fujisawa, K., Kodama, Y., Sato, M.: Performance of the supercomputer Fugaku for breadth-first search in Graph500 benchmark. In: Chamberlain, B.L., Varbanescu, AL., Ltaief, H., Luszczek, P. (eds.) High Performance Computing: 36th International Conference. ISC High Performance 2021, Virtual Event, 24 June–2 July 2021, Proceedings, pp. 372–390. Springer, Heidelberg (2021). https://doi.org/10.1007/978-3-030-78713-4_20

  20. Nehinbe, J.O.: A simple method for improving intrusion detections in corporate networks. In: Chamberlain, B.L., Varbanescu, AL., Ltaief, H., Luszczek, P. (eds.) Information Security and Digital Forensics: First International Conference, ISDF 2009, London, United Kingdom, 7–9 September 2009, Revised Selected Papers 1, pp. 111–122. Springer, Cham (2010). https://doi.org/10.1007/978-3-642-11530-1_13

  21. Reed, D.A., Dongarra, J.: Exascale computing and big data. Commun. ACM 58(7), 56–68 (2015)

    Article  Google Scholar 

  22. Sato, M., et al.: Co-design for A64FX manycore processor and “Fugaku”. In: SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–15 (2020). https://doi.org/10.1109/SC41405.2020.00051

  23. Shanbhag, S., Wolf, T.: Accurate anomaly detection through parallelism. IEEE Network 23(1), 22–28 (2009). https://doi.org/10.1109/MNET.2009.4804320

    Article  Google Scholar 

  24. Stojanovic, L., Dinic, M., Stojanovic, N., Stojadinovic, A.: Big-data-driven anomaly detection in industry (4.0): an approach and a case study. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 1647–1652. IEEE (2016)

    Google Scholar 

  25. Syarif, I., Zaluska, E., Prugel-Bennett, A., Wills, G.: Application of bagging, boosting and stacking to intrusion detection. In: Perner, P. (eds.) Machine Learning and Data Mining in Pattern Recognition: 8th International Conference, MLDM 2012, Berlin, Germany, 13–20 July 2012, Proceedings 8, pp. 593–602. Springer, Cham (2012). https://doi.org/10.1007/978-3-642-31537-4_46

  26. Ten, C.W., Hong, J., Liu, C.C.: Anomaly detection for cybersecurity of the substations. IEEE Trans. Smart Grid 2(4), 865–873 (2011). https://doi.org/10.1109/TSG.2011.2159406

    Article  Google Scholar 

  27. TOP500: Top500 list - November 2020 (2020). https://www.top500.org/lists/top500/2020/11/. Accessed 26 July 2023

  28. TOP500: Top500 list - June 2021 (2021). https://www.top500.org/lists/top500/2021/06/. Accessed 26 July 2023

  29. Ukil, A., Bandyoapdhyay, S., Puri, C., Pal, A.: IoT healthcare analytics: the importance of anomaly detection. In: 2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA), pp. 994–997. IEEE (2016)

    Google Scholar 

  30. Zineb, Z., Nahid, E., Ahmed, B.: A novel approach to parallel anomaly detection: application in cybersecurity. In: 2023 IEEE International Conference on Big Data (BigData), pp. 3574–3583. IEEE (2023)

    Google Scholar 

Download references

Acknowledgment

This research used computational resources of the supercomputer Fugaku provided by the RIKEN Center for Computational Science. We sincerely thank Research engineer Martial Mancip for his kind assistance, which greatly aided our study.

Author information

Authors and Affiliations

  1. University of Paris Saclay/Li-PaRAD/MDLS, Paris, France

    Zineb Ziani & Nahid Emad

  2. Numeryx, 17 Rue Jeanne Braconnier, 92360, Meudon, France

    Zineb Ziani

  3. RIKEN Center for Computational Science, Kobe, Hyogo, 650-0047, Japan

    Miwako Tsuji & Mitsuhisa Sato

Authors
  1. Zineb Ziani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nahid Emad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Miwako Tsuji
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mitsuhisa Sato
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zineb Ziani .

Editor information

Editors and Affiliations

  1. University of Malaga, Malaga, Spain

    Leonardo Franco

  2. University of Amsterdam, Amsterdam, The Netherlands

    Clélia de Mulatier

  3. AGH University of Science and Technology, Krakow, Poland

    Maciej Paszynski

  4. University of Amsterdam, Amsterdam, The Netherlands

    Valeria V. Krzhizhanovskaya

  5. University of Tennessee, Knoxville, TN, USA

    Jack J. Dongarra

  6. University of Amsterdam, Amsterdam, The Netherlands

    Peter M. A. Sloot

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ziani, Z., Emad, N., Tsuji, M., Sato, M. (2024). Enhancing the Parallel UC2B Framework: Approach Validation and Scalability Study. In: Franco, L., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2024. ICCS 2024. Lecture Notes in Computer Science, vol 14838. Springer, Cham. https://doi.org/10.1007/978-3-031-63783-4_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-63783-4_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-63785-8

  • Online ISBN: 978-3-031-63783-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation