Abstract
Context: A large number of source code metrics are reported in the literature. It is necessary to systematically collect, describe and classify source code metrics to support research and practice.
Objective: We aim to utilize existing secondary studies to develop a catalog of source code metrics together with their descriptions. The catalog will also provide information about which units of code (e.g., operators, operands, lines of code, variables, parameters, code blocks, or functions) are used to measure the internal quality attributes and the scope on which they are collected.
Method: We conducted a tertiary study to identify secondary studies reporting source code metrics. We have classified the source code metrics according to the measured internal quality attributes, the units of code used in the measures, and the scope at which the source code metrics are collected.
Results: From 711 secondary studies, we identified 52 relevant secondary studies. We reported 423 source code metrics together with their descriptions and the internal quality attributes they measure. Source code metrics predominantly incorporate function as a unit of code to measure internal quality attributes. In contrast, several source code metrics use more than one unit of code when measuring internal quality attributes. Nearly 51% of the source code metrics are collected at the class scope, while almost 12% and 15% of source code metrics are collected at module and application levels, respectively.
Conclusions: Researchers and practitioners can use the extensive catalog to assess which source code metrics meet their individual needs based on the description and classification scheme presented.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
The Centre for Reviews and Dissemination (CRD) Database of Abstracts of Reviews of Effects (DARE). https://www.crd.york.ac.uk/CRDWeb/. Accessed 20 Oct 2022
Ali, N.B., Petersen, K.: Evaluating strategies for study selection in systematic literature studies. In: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 1–4 (2014)
Ali, N.B., Tanveer, B.: A comparison of citation sources for reference and citation-based search in systematic literature reviews. e-informatica Soft. Eng. J. 16, 220106 (2022)
Ali, N.B., Usman, M.: Reliability of search in systematic reviews: towards a quality assessment framework for the automated-search strategy. Inf. Softw. Technol. 99, 133–147 (2018)
Ampatzoglou, A., Bibi, S., Avgeriou, P., Chatzigeorgiou, A.: Guidelines for managing threats to validity of secondary studies in software engineering. In: Contemporary Empirical Methods in Software Engineering, pp. 415–441. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-32489-6_15
Arisholm, E., Briand, L., Foyen, A.: Dynamic coupling measurement for object-oriented software. IEEE Trans. Softw. Eng. 30(8), 491–506 (2004)
Arvanitou, E., Ampatzoglou, A., Chatzigeorgiou, A., Galster, M., Avgeriou, P.: A map** study on design-time quality attributes and metrics. J. Syst. Softw. 127, 52–77 (2017)
Bandi, A., Williams, B., Allen, E.: Empirical evidence of code decay: a systematic map** study. In: Proceedings - Working Conference on Reverse Engineering, WCRE, pp. 341–350 (2013)
Bansiya, J., Davis, C.G.: A hierarchical model for object-oriented design quality assessment. IEEE Trans. Softw. Eng. 28(1), 4–17 (2002)
Barros-Justo, J.L., Benitti, F.B., Matalonga, S.: Trends in software reuse research: a tertiary study. Comput. Stand. Interfaces 66, 103352 (2019)
Briand, L.C., Wüst, J.: Empirical studies of quality models in object-oriented systems. Adv. Comput. 56, 97–166 (2002)
Budgen, D., Brereton, P., Williams, N., Drummond, S.: What support do systematic reviews provide for evidence-informed teaching about software engineering practice?. e-informatica Softw. Eng. J. 14(1), 7–60 (2020)
Caulo, M., Scanniello, G.: A taxonomy of metrics for software fault prediction. In: 2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 429–436. IEEE (2020)
Chen, L., Babar, M.A., Zhang, H.: Towards an evidence-based understanding of electronic data sources. In: 14th International Conference on Evaluation and Assessment in Software Engineering (EASE), pp. 1–4 (2010)
Chidamber, S.R., Kemerer, C.F.: A metrics suite for object oriented design. IEEE Trans. Softw. Eng. 20(6), 476–493 (1994)
Curcio, K., Santana, R., Reinehr, S., Malucelli, A.: Usability in agile software development: a tertiary study. Comput. Stand. Interfaces 64, 61–77 (2019)
El Emam, K.: Benchmarking kappa: interrater agreement in software process assessments. Empir. Softw. Eng. 4(2), 113–133 (1999)
El-Sharkawy, S., Yamagishi-Eichler, N., Schmid, K.: Metrics for analyzing variability and its implementation in software product lines: a systematic literature review. Inf. Softw. Technol. 106, 1–30 (2019)
Fenton, N., Bieman, J.: Software Metrics: A Rigorous and Practical Approach. CRC Press, Boca Raton (2019)
Halstead, M.H.: Elements of Software Science. Elsevier Science Ltd., Amsterdam (1977)
Hernandez-Gonzalez, E.Y., Sanchez-Garcia, A.J., Cortes-Verdin, M.K., Perez-Arriaga, J.C.: Quality metrics in software design: a systematic review. In: Proceedings of the 7th International Conference in Software Engineering Research and Innovation, pp. 80–86 (2019)
Hoda, R., Salleh, N., Grundy, J., Tee, H.M.: Systematic literature reviews in agile software development: a tertiary study. Inf. Softw. Technol. 85, 60–70 (2017)
Hosseini, S., Turhan, B., Gunarathna, D.: A systematic literature review and meta-analysis on cross project defect prediction. IEEE Trans. Softw. Eng. 45(2), 111–147 (2019)
Iftikhar, U., Ali, N.B., Börstler, J., Usman, M.: Dataset for a catalog of source code metrics - a tertiary study. https://doi.org/10.5281/zenodo.7219870. Accessed 20 Oct 2022
Iftikhar, U., Ali, N.B., Börstler, J., Usman, M.: A tertiary study on links between source code metrics and external quality attributes. Information and Software Technology (Submitted)
Jabangwe, R., Börstler, J., Šmite, D., Wohlin, C.: Empirical evidence on the link between object-oriented measures and external quality attributes: a systematic literature review. Empir. Softw. Eng. 20(3), 640–693 (2015)
Kaur, A.: A systematic literature review on empirical analysis of the relationship between code smells and software quality attributes. Arch. Comput. Methods Eng. 27(4), 1267–1296 (2019). https://doi.org/10.1007/s11831-019-09348-6
Kaur, A., Kaur, K., Pathak, K.: A proposed new model for maintainability index of open source software. In: Infocom Technologies and Optimization Proceedings of 3rd International Conference on Reliability, pp. 1–6 (2014)
Kaur, S., Singh, P.: How does object-oriented code refactoring influence software quality? Research landscape and challenges. J. Syst. Softw. 157, 110394 (2019)
Kitchenham, B.A., Budgen, D., Brereton, P.: Evidence-Based Software Engineering and Systematic Reviews, vol. 4. CRC Press, Boca Raton (2015)
Lacerda, G., Petrillo, F., Pimenta, M., Guéhéneuc, Y.G.: Code smells and refactoring: a tertiary systematic review of challenges and observations. J. Syst. Softw. 167, 110610 (2020)
Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33, 159–174 (1977)
Lanza, M., Marinescu, R.: Object-Oriented Metrics in Practice: Using Software Metrics to Characterize, Evaluate, and Improve the Design of Object-Oriented Systems. Springer, Cham (2007)
Li, W.: Another metric suite for object-oriented programming. J. Syst. Softw. 44(2), 155–162 (1998)
Malhotra, R., Chug, A.: Software maintainability: systematic literature review and current trends. Int. J. Softw. Eng. Knowl. Eng. 26(08), 1221–1253 (2016)
Martins, L., Afonso, P.J., Freire, A., Costa, H.: Evolution of quality assessment in SPL: a systematic map**. IET Softw. 14(6), 572–581 (2020)
McCabe, T.J.: A complexity measure. IEEE Trans. Softw. Eng. 2(4), 308–320 (1976)
de Normalización, O.I.: ISO-IEC 25010: 2011 systems and software engineering-systems and software quality requirements and evaluation (square)-system and software quality models. International Organization for Standardization, Geneva, Switzerland (2011)
Nuñez-Varela, A.S., Pérez-Gonzalez, H.G., Martínez-Perez, F.E., Soubervielle-Montalvo, C.: Source code metrics: a systematic map** study. J. Syst. Softw. 128, 164–197 (2017)
Petersen, K., Feldt, R., Mujtaba, S., Mattsson, M.: Systematic map** studies in software engineering. In: 12th International Conference on Evaluation and Assessment in Software Engineering (EASE) 12, pp. 1–10 (2008)
Petersen, K., Vakkalanka, S., Kuzniarz, L.: Guidelines for conducting systematic map** studies in software engineering: an update. Inf. Softw. Technol. 64, 1–18 (2015)
Saharudin, S., Wei, K., Na, K.: Machine learning techniques for software bug prediction: a systematic review. J. Comput. Sci. 16(11), 1558–1569 (2020)
Saraiva, J., et al.: Aspect-oriented software maintenance metrics: a systematic map** study. IET Semin. Digest 2012(1), 253–262 (2012)
Saraiva, J., Soares, S., Castor, F.: Towards a catalog of object-oriented software maintainability metrics. In: 2013 4th International Workshop on Emerging Trends in Software Metrics (WETSoM), pp. 84–87. IEEE, San Francisco, CA (2013)
Sharma, A., Dubey, S.K.: Comparison of software quality metrics for object-oriented system. Int. J. Comput. Sci. Manage. Stud. (IJCSMS) 12, 12–24 (2012)
Sreeji, K., Lakshmi, C.: A systematic literature review: recent trends and open issues in software refactoring. Int. J. Appl. Eng. Res. 10(18), 39696–39707 (2015)
Tran, H.K.V., Börstler, J., bin Ali, N., Unterkalmsteiner, M.: How good are my search strings? Reflections on using an existing review as a quasi-gold standard. e-Informatica Softw. Eng. J. 16(1), 220103 (2022)
Turner, M.: Digital libraries and search engines for software engineering research: an overview. Keele University, UK (2010)
Vogel, M., et al.: Metrics in automotive software development: a systematic literature review. J. Softw. Evol. Process 33(2), e2296 (2021)
Yan, M., ** study. Sci. China Inf. Sci. 62(9), 1–18 (2019). https://doi.org/10.1007/s11432-018-9608-3
Yan, M., ** study of quality assessment models for software products. In: 2017 International Conference on Software Analysis, Testing and Evolution (SATE), pp. 63–71. IEEE (2017)
Acknowledgment
This work has been supported by ELLIIT, a Strategic Area within IT and Mobile Communications, funded by the Swedish Government. The work has also been supported by the OSIR project funded by the Swedish Knowledge Foundation (grant number 20190081).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Iftikhar, U., Ali, N.B., Börstler, J., Usman, M. (2023). A Catalog of Source Code Metrics – A Tertiary Study. In: Mendez, D., Winkler, D., Kross, J., Biffl, S., Bergsmann, J. (eds) Software Quality: Higher Software Quality through Zero Waste Development. SWQD 2023. Lecture Notes in Business Information Processing, vol 472. Springer, Cham. https://doi.org/10.1007/978-3-031-31488-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-31488-9_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31487-2
Online ISBN: 978-3-031-31488-9
eBook Packages: Computer ScienceComputer Science (R0)