Abstract
With the rapid growth of microservice systems in cloud-native environments, end-to-end traces have become essential data to help diagnose performance issues. However, existing trace-based anomaly detection and root cause analysis (RCA) still suffer from practical issues due to either the massive volume or frequent system changes. In this study, we propose a lightweight and adaptive trace-based anomaly detection and RCA approach, named MicroSketch, which leverages Sketch based features and Robust Random Cut Forest (RRCForest) to render trace analysis more effective and efficient. In addition, MicroSketch is an unsupervised approach that is able to adapt to changes in microservice systems without any human intervention. We evaluated MicroSketch on a widely-used open-source system and a production system. The results demonstrate the efficiency and effectiveness of MicroSketch. MicroSketch significantly outperforms start-of-the-art approaches, with an average of 40.9% improvement in F1 score on anomaly detection and 25.0% improvement in Recall of Top-1 on RCA. In particular, MicroSketch is at least 60x faster than other methods in terms of diagnosis time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Jaeger, https://jaegertracing.io/.
- 2.
Zipkin, https://zipkin.io/.
- 3.
Chaosblade, https://github.com/chaosblade-io/chaosblade.
- 4.
Strace, https://strace.io.
References
Chen, P., Qi, Y., et al.: Causeinfer: automatic and distributed performance diagnosis with hierarchical causality graph in large distributed systems. In: INFOCOM 2014, pp. 1887–1895. IEEE (2014)
Dragoni, N., et al.: Microservices: yesterday, today, and tomorrow. In: Present and Ulterior Software Engineering, pp. 195–216. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67425-4_12
Gan, Y., Zhang, Y., et al.: Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices. In: ASPLOS, pp. 19–33 (2019)
Gao, K., Sun, C., et al., S.W.: Buffer-based end-to-end request event monitoring in the cloud. In: NSDI 22, pp. 829–843. USENIX Association (2022)
Guha, S., Mishra, N., et al.: Robust random cut forest based anomaly detection on streams. In: ICML, pp. 2712–2721. PMLR (2016)
Huang, L., Zhu, T.: tprof: performance profiling via structural aggregation and automated analysis of distributed systems traces. In: SoCC 2021, pp. 76–91. ACM (2021)
Kim, M., Sumbaly, R., et al.: Root cause detection in a service-oriented architecture. ACM SIGMETRICS Perform. Eval. Rev. 41(1), 93–104 (2013)
Lin, J., Chen, P., Zheng, Z.: Microscope: pinpoint performance issues with causal graphs in micro-service environments. In: Pahl, C., Vukovic, M., Yin, J., Yu, Q. (eds.) ICSOC 2018. LNCS, vol. 11236, pp. 3–20. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03596-9_1
Liu, F.T., Ting, K.M., et al.: Isolation-based anomaly detection. TKDD 6(1), 1–39 (2012)
Liu, P., Xu, H., et al.: Unsupervised detection of microservice trace anomalies through service-level deep bayesian networks. In: ISSRE 2020, pp. 48–58. IEEE (2020)
Masson, C., Rim, J.E., et al.: DDSketch: a fast and fully-mergeable quantile sketch with relative-error guarantees. Proc. VLDB Endow. 12(12), 2195–2205 (2019)
Nedelkoski, S., Cardoso, J., Kao, O.: Anomaly detection from system tracing data using multimodal deep learning. In: CLOUD 2019, pp. 179–186. IEEE (2019)
Pitakrat, T., Okanović, D., et al.: Hora: architecture-aware online failure prediction. JSE 137, 669–685 (2018)
Shkuro, Y.: Mastering Distributed Tracing: Analyzing performance in Microservices and Complex Systems. Packt Publishing Ltd, Birmingham (2019)
Sigelman, B.H., Barroso, L.A., et al.: Dapper, a large-scale distributed systems tracing infrastructure. Google, Inc, Technical Report (2010)
Soldani, J., Tamburriand, et al.: The pains and gains of microservices: a systematic grey literature review. J. Syst. Softw. 146, 215–232 (2018)
Thalheim, J., Bhatotia, P., et al.: Cntr: Lightweight \(\{\)OS\(\}\) containers. In: 2018 USENIX, pp. 199–212 (2018)
Yu, G., Chen, P., et al.: Microrank: end-to-end latency issue localization with extended spectrum analysis in microservice environments. In: WWW 2021, pp. 3087–3098. ACM / IW3C2 (2021)
Yu, G., Chen, P., Zheng, Z.: Microscaler: automatic scaling for microservices with an online learning approach. In: ICWS 2019, pp. 68–75. IEEE (2019)
Yu, G., Chen, P., Zheng, Z.: Microscaler: cost-effective scaling for microservice applications in the cloud with an online learning approach. IEEE TCC 10(2), 1100–1116 (2022)
Zhou, X., Peng, X., et al.: Fault analysis and debugging of microservice systems: industrial survey, benchmark system, and empirical study. TSE 47(2), 243–260 (2018)
Acknowledgements
The research is supported by the National Key Research and Development Program of China (2019YFB1804002), the National Natural Science Foundation of China (No. 62272495, 61902440), the Basic and Applied Basic Research of Guangzhou (No. 202002030328), and the Natural Science Foundation of Guangdong Province (No. 2019A1515012229). The corresponding author is Pengfei Chen.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Y., Yu, G., Chen, P., Zhang, C., Zheng, Z. (2022). MicroSketch: Lightweight and Adaptive Sketch Based Performance Issue Detection and Localization in Microservice Systems. In: Troya, J., Medjahed, B., Piattini, M., Yao, L., Fernández, P., Ruiz-Cortés, A. (eds) Service-Oriented Computing. ICSOC 2022. Lecture Notes in Computer Science, vol 13740. Springer, Cham. https://doi.org/10.1007/978-3-031-20984-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-20984-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20983-3
Online ISBN: 978-3-031-20984-0
eBook Packages: Computer ScienceComputer Science (R0)