SpringerLink
Log in

Unsupervised Anomaly Detection Process Using LLE and HDBSCAN by Style-GAN as a Feature Extractor

  • Regular Paper
  • Published:
International Journal of Precision Engineering and Manufacturing Aims and scope Submit manuscript

Abstract

This study proposes a process for detecting anomalies in the manufacturing industry, where data imbalance is a frequent problem. The labeling of anomalies can be challenging owing to the different types of anomalies. To address this issue, we used clustering based on the distribution of acquired normal data. We extracted latent vector values from normal image data as features using the Style-GAN method, after conversion of the time-series data. Subsequently, we performed dimensionality reduction through Locally Linear Embedding (LLE) using the extracted latent vector values and selected the Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) for anomaly detection. We verified the proposed process using a milling dataset that included measurements of vibration, force, and noise. The evaluation of the process included dimensionality reduction methods such as Locally Linear Embedding (LLE), Principal Component Analysis (PCA), Kernel PCA, Singular Value Decomposition (SVD), and ISOmetric map** (ISO) produced an F-1 score of 0.86.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Availability of data and materials

“Not applicable”

Code availability

Not applicable

References

  1. Selvaraj, V., & Min, S. (2023). Real-time fault identification system for a retrofitted ultra-precision CNC machine from equipment’s power consumption data: A case study of an implementation. International Journal of Precision Engineering and Manufacturing-Green Technology. https://doi.org/10.1007/s40684-022-00497-x

    Article  Google Scholar 

  2. Nam, J. S., & Kwon, W. T. (2022). A study on tool breakage detection during milling process using LSTM-autoencoder and gaussian mixture model. International Journal of Precision Engineering and Manufacturing, 23(6), 667–675.

    Article  Google Scholar 

  3. Svinth, C. N., Wallace, S., Kim, T. G., et al. (2022). Identifying abnormal CFRP holes using both unsupervised and supervised learning techniques on in-process force, current, and vibration signals. International Journal of Precision Engineering and Manufacturing, 23(6), 609–625.

    Article  Google Scholar 

  4. Kim, H. S., Han, Y., & Kim, J. (2023). 3D measurement using a single image for smart manufacturing of microscopic products in a ceramic powder pressing process. International Journal of Precision Engineering and Manufacturing-Green Technology, 10(1), 233–243.

    Article  Google Scholar 

  5. Ferdousi, Z., & Maeda, A. (2006). Unsupervised outlier detection in time series data. In 22nd international conference on data engineering workshops (ICDEW’06) (pp. x121–x121), IEEE.

  6. Kamat, P., & Sugandhi, R. (2020). Anomaly detection for predictive maintenance in industry 4.0-A survey. In E3S web of conferences (Vol. 170, p. 02007). EDP Sciences. variousfactors.

  7. Kim, S. W., Kong, J. H., Lee, S. W., & Lee, S. (2022). Recent advances of artificial intelligence in manufacturing industrial sectors: A review. International Journal of Precision Engineering and Manufacturing. https://doi.org/10.1007/s12541-021-00600-3

    Article  Google Scholar 

  8. Kim, J, et al. (2020). GAN-based anomaly detection in imbalance problems. Computer Vision-ECCV 2020 Workshops: Glasgow, UK, August 23-28, 2020, Proceedings, Part VI 16. Springer International Publishing.

  9. **a, Xuan, et al. (2022). GAN-based anomaly detection: A review. Neurocomputing, 493, 497–535.

    Article  Google Scholar 

  10. Son, J., Kim, C., & Jeong, M. (2022). Unsupervised learning for anomaly detection of electric motors. International Journal of Precision Engineering and Manufacturing, 23(4), 421–427.

    Article  Google Scholar 

  11. Kwon, Y. W., & Kang, D. J. (2023). Anomaly detecting geometric transformation network with outlier exposure defect inspection of real industrial data. International Journal of Precision Engineering and Manufacturing, 24(1), 73–81.

    Article  Google Scholar 

  12. Agrawal, S., & Agrawal, J. (2015). Survey on anomaly detection using data mining techniques. Procedia Computer Science, 60, 708–713.

    Article  Google Scholar 

  13. Pang, G., et al. (2021). Deep learning for anomaly detection: A review. ACM Computing Surveys (CSUR), 54(2), 1–38.

    Article  Google Scholar 

  14. Pang, G., Cao, L., & Aggarwal, C. (2021). Deep learning for anomaly detection: Challenges, methods, and opportunities. In Proceedings of the 14th ACM international conference on web search and data mining.

  15. Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys (CSUR), 41(3), 1–58.

    Article  Google Scholar 

  16. Chalapathy, R., & Chawla, S. (2019). Deep learning for anomaly detection: A survey. ar**v preprint ar**v:1901.03407.

  17. Omar, S., Ngadi, A., & Jebur, H. H. (2013). Machine learning techniques for anomaly detection: An overview. International Journal of Computer Applications, 79(2).

  18. Wang, Z., & Oates, T. (2015). Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. In Workshops at the twenty-ninth AAAI conference on artificial intelligence (Vol. 1). Menlo Park: AAAI.

  19. LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989). Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4), 541–551.

    Article  Google Scholar 

  20. Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 1440–1448).

  21. Faouzi, J., & Janati, H. (2020). pyts: A python package for time series classification. The Journal of Machine Learning Research, 21(1), 1720–1725.

    Google Scholar 

  22. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., & Bengio, Y. (2020). Generative adversarial networks. Communications of the ACM, 63(11), 139–144.

    Article  MathSciNet  Google Scholar 

  23. **a, X., Pan, X., Li, N., He, X., Ma, L., Zhang, X., & Ding, N. (2022). GAN-based anomaly detection: A review. Neurocomputing, 493, 497–535.

    Article  Google Scholar 

  24. Karras, T., Laine, S., & Aila, T. (2019). A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.

  25. Fodor, I. K. (2002). A survey of dimension reduction techniques (No. UCRL-ID-148494). Lawrence Livermore National Lab.

    Book  Google Scholar 

  26. Dunteman, G. H. (1989). Principal components analysis (No. 69). Sage.

    Book  Google Scholar 

  27. Schölkopf, B., Smola, A., & Müller, K.-R. (1997). Kernel principal component analysis. International conference on artificial neural networks. Springer.

    Google Scholar 

  28. Roweis, S. T. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500), 2323–2326.

    Article  Google Scholar 

  29. Balasubramanian, M., & Schwartz, E. L. (2002). The isomap algorithm and topological stability. Science, 295(5552), 7–7.

    Article  Google Scholar 

  30. Xu, R., & Wunsch, D. (2005). Survey of clustering algorithms. IEEE Transactions on Neural Networks, 16(3), 645–678.

    Article  Google Scholar 

  31. Xu, D., & Tian, Y. (2015). A comprehensive survey of clustering algorithms. Annals of Data Science, 2, 165–193.

    Article  Google Scholar 

  32. Hartigan, J. A., & Wong, M. A. (1979). Algorithm AS 136: A k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 28(1), 100–108.

    Google Scholar 

  33. Comaniciu, D., & Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5), 603–619.

    Article  Google Scholar 

  34. Rasmussen, C. (1999). The infinite Gaussian mixture model. Advances in Neural Information Processing Systems, 12.

  35. Reynolds, D. A. (2009). Gaussian mixture models. Encyclopedia of Biometrics, 741, 659–663.

    Article  Google Scholar 

  36. Ester, M., Kriegel, H. P., Sander, J., & Xu, X. (1996, August). A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd (Vol. 96, No. 34, pp. 226-231).

  37. Campello, R. J., Moulavi, D., & Sander, J. (2013). Density-based clustering based on hierarchical density estimates. In Advances in knowledge discovery and data mining: 17th pacific-asia conference, PAKDD 2013, Gold Coast, Australia, April 14–17, 2013, Proceedings, Part II 17 (pp. 160-172). Springer Berlin Heidelberg.

  38. McInnes, L., Healy, J., & Astels, S. (2017). hdbscan: Hierarchical density based clustering. Journal of Open Source Software, 2(11), 205.

    Article  Google Scholar 

  39. PHM Society. (2010). PHM data chalelnge 2010.

  40. Martínez-Arellano, G., Terrazas, G., & Ratchev, S. (2019). Tool wear classification using time series imaging and deep learning. The International Journal of Advanced Manufacturing Technology, 104, 3647–3662.

    Article  Google Scholar 

Download references

Funding

This study has been conducted with the support of the Ministry of SMEs and Startups as “Development of intelligent SHWIS (AI - Smart Human Work Interactive Interface System) AR technology that provides AR Inspection (2/4)- RS-2022-00140809”, and supported by Korea Institute for Advancement of Technology(KIAT) grant funded by the Korea Government as “Tools for adaptive and intelligent control of discrete manufacturing process (TANDEM) (2/4)- P0022309.

Author information

Authors and Affiliations

  1. Korea Institute of Industrial Technology (KITECH), 89, Yangdaegiro-gil, Ipjang-myeon, Seobuk-gu, Cheonan-si, Chungcheongnam-do, Republic of Korea

    Taeheon Lee, Yoonseok Kim, Youngjoo Hyun & Youngjun Yoo

  2. Department of Information and Industrial Engineering, Yonsei University, Seoul, Republic of Korea

    Taeheon Lee, Yoonseok Kim, Youngjoo Hyun & Jeonghoon Mo

Authors
  1. Taeheon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yoonseok Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Youngjoo Hyun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeonghoon Mo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Youngjun Yoo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jeonghoon Mo or Youngjun Yoo.

Ethics declarations

Consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

The author declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, T., Kim, Y., Hyun, Y. et al. Unsupervised Anomaly Detection Process Using LLE and HDBSCAN by Style-GAN as a Feature Extractor. Int. J. Precis. Eng. Manuf. 25, 51–63 (2024). https://doi.org/10.1007/s12541-023-00908-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12541-023-00908-2

Keywords

Search

Navigation