SpringerLink
Log in

Unsupervised diffusion based anomaly detection for time series

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Unsupervised anomaly detection aims to construct a model that effectively detects invisible anomalies by training and reconstruct normal data. While a significant amount of reconstruction-based methods has made effective progress for time series anomaly detection, challenges still exist in aspects such as temporal feature extraction and generalization ability. Firstly, temporal features of data are subject to local information interference in reconstruction methods, which limits the long-term signal reconstruction methods. Secondly, the training dataset collector is subject to information nourishment such as collection methods, collection periods and locations, and data patterns are diverse, requiring the model to rebuild normal data according to different patterns. These issues hinder the anomaly detection capability of reconstruction-based methods. We propose an unsupervised anomaly detection model based on a diffusion model, which learns normal data pattern learning through noisy forward diffusion and reverse noise regression. By using a cascaded structure and combining it with a structured state space layer, long-term time series signal feature can be well extracted. Different collection signals are distinguished by introducing collector entity ID embedding. The method proposed in this article significantly improves performance in experimental tests on three public datasets. Innovative aspects: (1) Utilizing the S4 method to capture long-term dependencies; (2) Employing a diffusion model for reconstruction learning; (3) Leveraging embedding techniques to enhance different pattern learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Pimentel MA, Clifton DA, Clifton L, Tarassenko L (2014) A review of novelty detection. Signal Process 99:215–249

    Article  Google Scholar 

  2. Schmidl S, Wenig P, Papenbrock T (2022) Anomaly detection in time series: a comprehensive evaluation. Proceedings of the VLDB Endowment 15(9):1779–1797

    Article  Google Scholar 

  3. Shaukat K, Alam TM, Luo S, Shabbir S, Hameed IA, Li J, Abbas SK, Javed U (2021) A review of time-series anomaly detection techniques: a step to future perspectives. In: Advances in information and communication: proceedings of the 2021 future of information and communication conference (FICC), Springer, vol 1, pp 865–877 (2021)

  4. Blázquez-García A, Conde A, Mori U, Lozano JA (2021) A review on outlier/anomaly detection in time series data. ACM Comput. Surveys (CSUR) 54(3):1–33

    Article  Google Scholar 

  5. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv 41:15

    Article  Google Scholar 

  6. Ho J, Jain A, Abbeel P (2020) Denoising diffusion probabilistic models. Adv Neural Inform Process Syst 33:6840–6851

    Google Scholar 

  7. Li Y, Lu X, Wang Y, Dou D (2022) Generative time series forecasting with diffusion, denoise, and disentanglement. Adv Neural Inform Process Syst 35:23009–23022

    Google Scholar 

  8. Hayes MA, Capretz MA (2015) Contextual anomaly detection framework for big sensor data. J Big Data 2(1):1–22

  9. Kumari R, Singh M, Jha R, Singh N et al (2016) Anomaly detection in network traffic using k-mean clustering. In: 2016 3rd International conference on recent advances in information technology (RAIT), IEEE, pp 387–393

  10. Ramaswamy S, Rastogi R, Shim K (2000) Efficient algorithms for mining outliers from large data sets. In: Proceedings of the 2000 ACM SIGMOD international conference on management of data, pp 427–438

  11. Breunig MM, Kriegel H-P, Ng RT, Sander J (2000) Lof: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD international conference on management of data, pp 93–104 (2000)

  12. Perer A, Sun J (2012) Matrixflow: temporal network visual analytics to track symptom evolution during disease progression. In: AMIA annual symposium proceedings, American Medical Informatics Association, vol 2012, p 716

  13. Su Y, Zhao Y, Niu C, Liu R, Sun W, Pei D (2019) Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: Proceedings of the 25th ACM SIGKDD International conference on knowledge discovery & data mining, pp 2828–2837

  14. Malhotra P, Vig L, Shroff G, Agarwal P et al (2015) Long short term memory networks for anomaly detection in time series. In: ESANN, vol 2015, p 89

  15. Zhou B, Liu S, Hooi B, Cheng X, Ye J (2019) Beatgan: anomalous rhythm detection using adversarially generated time series. IJCAI 2019:4433–4439

    Google Scholar 

  16. Li L, Yan J, Wen Q, ** Y, Yang X (2022) Learning robust deep state space for unsupervised anomaly detection in contaminated time-series. IEEE Trans Knowl Data Eng

  17. Wang F, Wang K, Yao B (2023) Time series anomaly detection with reconstruction-based state-space models. In: International conference on artificial neural networks, Springer, pp 74–86

  18. Li C-L, Sohn K, Yoon J, Pfister T (2021) Cutpaste: self-supervised learning for anomaly detection and localization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9664–9674

  19. Hayashi T, Cimr D, Studnička F, Fujita H, Bušovskỳ D, Cimler R, Selamat A (2024) Distance-based one-class time-series classification approach using local cluster balance. Expert Syst Appl 235:121201

    Article  Google Scholar 

  20. Hayashi T, Cimr D, Studnička F, Fujita H, Bušovskỳ D, Cimler R (2024) Patient deterioration detection using one-class classification via cluster period estimation subtask. Inform Sci 657:119975

    Article  Google Scholar 

  21. Lenz OU, Peralta D, Cornelis C (2021) Average localised proximity: a new data descriptor with good default one-class classification performance. Pattern Recognit 118:107991

    Article  Google Scholar 

  22. Pinaya WH, Graham MS, Gray R, Da Costa PF, Tudosiu P-D, Wright P, Mah YH, MacKinnon AD, Teo JT, Jager R et al (2022) Fast unsupervised brain anomaly detection and segmentation with diffusion models. In: Part VIII (ed) Medical image computing and computer assisted intervention-MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings. Springer, pp 705–714

    Google Scholar 

  23. Tashiro Y, Song J, Song Y, Ermon S (2021) Csdi: conditional score-based diffusion models for probabilistic time series imputation. Adv Neural Inform Process Syst 34:24804–24816

    Google Scholar 

  24. Alcaraz JML, Strodthoff N (2022) Diffusion-based time series imputation and forecasting with structured state space models. ar**v:2208.09399

  25. Harvey W, Naderiparizi S, Masrani V, Weilbach C, Wood F (2022) Flexible diffusion modeling of long videos. Adv Neural Inform Process Syst 35:27953–27965

    Google Scholar 

  26. **ao C, Gou Z, Tai W, Zhang K, Zhou F (2023) Imputation-based time-series anomaly detection with conditional weight-incremental diffusion models. In: Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, pp 2742–2751

  27. Muñoz-Gil G, Corominas GG, Lewenstein M (2021) Unsupervised learning of anomalous diffusion data: an anomaly detection approach. J Phys A: Mathematical Theoretical 54(50):504001

    Article  MathSciNet  Google Scholar 

  28. Sperl RE, Chung SM (2019) Two-step anomaly detection for time series data. In: 2019 International conference on data and software engineering (ICoDSE), IEEE, pp 1–5

  29. Li J, Di S, Shen Y, Chen L (2021) Fluxev: a fast and effective unsupervised framework for time-series anomaly detection. In: Proceedings of the 14th ACM international conference on web search and data mining, pp 824–832

  30. Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B (2022) High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10684–10695

  31. Kong Z, ** W, Huang J, Zhao K, Catanzaro B (2020) Diffwave: a versatile diffusion model for audio synthesis. In: International conference on learning representations

  32. Hoogeboom E, Nielsen D, Jaini P, Forré P, Welling M (2021) Argmax flows and multinomial diffusion: learning categorical distributions. Adv Neural Inform Process Syst 34:12454–12465

    Google Scholar 

  33. Kitagawa G (1998) A self-organizing state-space model. J Am Stat Assoc 1203–1215

  34. Krishnan R, Shalit U, Sontag D (2017) Structured inference networks for nonlinear state space models. In: Proceedings of the AAAI conference on artificial intelligence, vol 31

  35. Smith JT, Warrington A, Linderman S (2022) Simplified state space layers for sequence modeling. In: The eleventh international conference on learning representations

  36. Oyedotun OK, Al Ismaeil K, Aouada D (2021) Training very deep neural networks: rethinking the role of skip connections. Neurocomputing 441:105–117

    Article  Google Scholar 

  37. Hundman K, Constantinou V, Laporte C, Colwell I, Soderstrom T (2018) Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 387–395

  38. Simoliuniene R, Krisciukaitis A, Macas A, Baksyte G, Saferis V, Zaliunas R (2008) Principal component analysis based method for detection and evaluation of ecg t-wave alternans. In: 2008 Computers in cardiology, IEEE, pp 757–760

  39. Zhang R, Zhang S, Muthuraman S, Jiang J (2007) One class support vector machine for anomaly detection in the communication network performance data. In: Proceedings of the 5th conference on applied electromagnetics, wireless and optical communications, Citeseer, pp 31–37

  40. Provotar OI, Linder YM, Veres MM (2019) Unsupervised anomaly detection in time series using lstm-based autoencoders. In: 2019 IEEE International conference on advanced trends in information theory (ATIT), IEEE, pp 513–517

  41. Li Y, Li K, Chen C, Zhou X, Zeng Z, Li K (2021) Modeling temporal patterns with dilated convolutions for time-series forecasting. ACM Trans Knowl Discovery Data (TKDD) 16(1):1–22

    Google Scholar 

  42. De Vazelhes W, Carey C, Tang Y, Vauquier N, Bellet A (2020) metric-learn: metric learning algorithms in python. J Mach Learn Res 21(1):5447–5452

    Google Scholar 

  43. Harrou F, Kadri F, Chaabane S, Tahon C, Sun Y (2015) Improved principal component analysis for anomaly detection: application to an emergency department. Comput Indust Eng 88:63–77

    Article  Google Scholar 

  44. Měkota O (2019) Anomaly detection using generative adversarial networks

  45. Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G (2017) Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Information processing in medical imaging: 25th International Conference, IPMI 2017, Boone, NC, USA, June 25-30, 2017, Proceedings, Springer, pp 146–157

Download references

Author information

Author notes

  1. Haiwei Zuo, Aiqun Zhu, Yin** Liao, Shiman Li and Yun Chen contributed equally to this work.

Authors and Affiliations

  1. Department of Big Data Application Technology, School of Information and Communication Technology, City Polytechnic of Shenzhen (Shenzhen Institute of Technology), No.1 Jiangjunmao Road, Shenzhen, 518000, Guangdong, China

    Haiwei Zuo, Aiqun Zhu, Yan** Zhu, Yin** Liao, Shiman Li & Yun Chen

  2. Faculty of Engineering and Science, Curtin University Malaysia, CDT 250, Miri, 98009, Sarawak, Malaysia

    Haiwei Zuo

  3. Research Administration Office, City Polytechnic of Shenzhen (Shenzhen Institute of Technology), No.1 Jiangjunmao Road, Shenzhen, 518000, Guangdong, China

    Aiqun Zhu

Authors
  1. Haiwei Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aiqun Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan** Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yin** Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shiman Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan** Zhu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zuo, H., Zhu, A., Zhu, Y. et al. Unsupervised diffusion based anomaly detection for time series. Appl Intell (2024). https://doi.org/10.1007/s10489-024-05341-0

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05341-0

Keywords

Search

Navigation