SpringerLink
Log in

End-to-End Test Coverage Metrics in Microservice Systems: An Automated Approach

  • Conference paper
  • First Online:
Service-Oriented and Cloud Computing (ESOCC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14183))

Included in the following conference series:

  • 352 Accesses

Abstract

Microservice architecture gains momentum by fueling systems with cloud-native benefits, scalability, and decentralized evolution. However, new challenges emerge for end-to-end (E2E) testing. Testers who see the decentralized system through the user interface might assume their tests are comprehensive, covering all middleware endpoints scattered across microservices. However, they do not have instruments to verify such assumptions. This paper introduces test coverage metrics for evaluating the extent of E2E test suite coverage for microservice endpoints. Next, it presents an automated approach to compute these metrics to provide feedback on the completeness of E2E test suites. Furthermore, a visual perspective is provided to highlight test coverage across the system’s microservices to guide on gaps in test suites. We implement a proof-of-concept tool and perform a case study on a well-established system benchmark showing it can generate conclusive feedback on test suite coverage over system endpoints.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Swagger https://swagger.io.

  2. 2.

    Base64: https://developer.mozilla.org/en-US/docs/Glossary/Base64.

  3. 3.

    Prototype: https://github.com/cloudhubs/test-coverage-backend.

  4. 4.

    JavaParser: https://github.com/javaparser/javaparser.

  5. 5.

    ELK: https://aws.amazon.com/what-is/elk-stack.

  6. 6.

    Elasticsearch Java Client: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high.html.

  7. 7.

    Coverage Visualizer: https://github.com/cloudhubs/test-coverage-frontend.

  8. 8.

    Prophet: https://github.com/cloudhubs/graal-prophet-utils.

  9. 9.

    3D Visualizer: https://github.com/cloudhubs/graal_mvp.

  10. 10.

    SkyWalking: https://skywalking.apache.org/docs.

  11. 11.

    TrainTicket: https://github.com/cloudhubs/train-ticket/tree/v1.0.1.

  12. 12.

    Test benchmark: https://github.com/cloudhubs/microservice-tests.

  13. 13.

    Dataset: https://zenodo.org/record/8055457.

References

  1. Corradini, D., Zampieri, A., Pasqua, M., Ceccato, M.: Empirical comparison of black-box test case generation tools for restful APIs. In: 2021 IEEE 21st International Working Conference on Source Code Analysis and Manipulation (SCAM), pp. 226–236 (2021). https://doi.org/10.1109/SCAM52516.2021.00035

  2. FudanSELab: Home. https://github.com/FudanSELab/train-ticket/wiki

  3. Ghani, I., Wan-Kadir, W.M., Mustafa, A., Imran Babir, M.: Microservice testing approaches: a systematic literature review. Int. J. Integr. Eng. 11(8), 65–80 (2019). https://publisher.uthm.edu.my/ojs/index.php/ijie/article/view/3856

  4. Giamattei, L., Guerriero, A., Pietrantuono, R., Russo, S.: Automated grey-box testing of microservice architectures. In: 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS), pp. 640–650 (2022)

    Google Scholar 

  5. Jiang, P., Shen, Y., Dai, Y.: Efficient software test management system based on microservice architecture. In: 2022 IEEE 10th Joint International Information Technology and Artificial Intelligence Conference, vol. 10, pp. 2339–2343 (2022)

    Google Scholar 

  6. Ma, S.P., Fan, C.Y., Chuang, Y., Lee, W.T., Lee, S.J., Hsueh, N.L.: Using service dependency graph to analyze and test microservices. In: 2018 IEEE 42nd Annual Computer Software and Applications Conference, vol. 2, pp. 81–86 (2018)

    Google Scholar 

  7. Schiewe, M., Curtis, J., Bushong, V., Cerny, T.: Advancing static code analysis with language-agnostic component identification. IEEE Access 10, 30743–30761 (2022). https://doi.org/10.1109/ACCESS.2022.3160485

    Article  Google Scholar 

  8. Smith, S., et al.: Benchmarks for end-to-end microservices testing (2023)

    Google Scholar 

  9. Waseem, M., Liang, P., Shahin, M., Di Salle, A., Márquez, G.: Design, monitoring, and testing of microservices systems: the practitioners’ perspective. J. Syst. Softw. 182, 111061 (2021)

    Article  Google Scholar 

  10. Wohlin, C., Runeson, P., Hst, M., Ohlsson, M.C., Regnell, B., Wessln, A.: Experimentation in Software Engineering. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29044-2

    Book  Google Scholar 

  11. Zhao, X., et al.: lprof: a non-intrusive request flow profiler for distributed systems. In: 11th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation, pp. 629–644 (2014)

    Google Scholar 

Download references

Acknowledgements

This material is supported by the National Science Foundation under Grant No. 2245287 and Grant No. 349488 (MuFAno) from the Academy of Finland.

Author information

Authors and Affiliations

  1. Computer Science, Baylor University, One Bear Place, Waco, TX, 97141, USA

    Amr S. Abdelfattah, Jorge Yero Salazar, Austin Lehman, Joshua Hunter & Ashley Bickham

  2. Systems and Industrial Engineering, University of Arizona, Tucson, AZ, USA

    Tomas Cerny

  3. University of Oulu, Oulu, Finland

    Davide Taibi

Authors
  1. Amr S. Abdelfattah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tomas Cerny
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jorge Yero Salazar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Austin Lehman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Joshua Hunter
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ashley Bickham
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Davide Taibi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomas Cerny .

Editor information

Editors and Affiliations

  1. University of Cyprus, Nicosia, Cyprus

    George A. Papadopoulos

  2. RWTH Aachen University, Aachen, Germany

    Florian Rademacher

  3. University of Pisa, Pisa, Italy

    Jacopo Soldani

Rights and permissions

Reprints and permissions

Copyright information

© 2023 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Abdelfattah, A.S. et al. (2023). End-to-End Test Coverage Metrics in Microservice Systems: An Automated Approach. In: Papadopoulos, G.A., Rademacher, F., Soldani, J. (eds) Service-Oriented and Cloud Computing. ESOCC 2023. Lecture Notes in Computer Science, vol 14183. Springer, Cham. https://doi.org/10.1007/978-3-031-46235-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-46235-1_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-46234-4

  • Online ISBN: 978-3-031-46235-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation