Abstract
Anyone working in the technology sector is probably familiar with the question: “Have you tried turning it off and on again?”, as this is usually the default question asked by tech support. Similarly, it is known in search-based testing that metaheuristics might get trapped in a plateau during a search. As a human, one can look at the gradient of the fitness curve and decide to restart the search, so as to hopefully improve the results of the optimization with the next run. Trying to automate such a restart, it has to be programmatically decided whether the metaheuristic has encountered a plateau yet, which is an inherently difficult problem. To mitigate this problem in the context of theoretical search problems, the Bet and Run strategy was developed, where multiple algorithm instances are started concurrently, and after some time all but the single most promising instance in terms of fitness values are killed. In this paper, we adopt and evaluate the Bet and Run strategy for the problem of test case generation. Our work indicates that use of this restart strategy does not generally lead to gains in the quality metrics, when instantiated with the best parameters found in the literature.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Download of EvoSuite after commit https://github.com/EvoSuite/evosuite/commit/e26a6b725539370aa22976f041f3304972e201a1 on January 10, 2020.
- 2.
- 3.
- 4.
References
Baars, A., et al.: Symbolic search-based testing. In: Proceedings of the 2011 26th International Conference on Automated Software Engineering, ASE 2011, pp. 53–62. IEEE (2011)
Bianchi, L., Dorigo, M., Gambardella, L.M., Gutjahr, W.J.: A survey on metaheuristics for stochastic combinatorial optimization. Nat. Comput. Int. J. 8(2), 239–287 (2009)
Chan, K.P., Chen, T.Y., Towey, D.: Forgetting test cases. In: Proceedings of the 30th International Computer Software and Applications Conference, COMPSAC 2006, vol. 01, pp. 485–494. IEEE (2006)
Fischetti, M., Monaci, M.: Exploiting erraticism in search. Oper. Res. 62(1), 114–122 (2014)
Fraser, G., Arcuri, A.: EvoSuite: automatic test suite generation for object-oriented software. In: 19th Symposium on the Foundations of Software Engineering (FSE) and 13th European Software Engineering Conference (ESEC), pp. 416–419. ACM (2011)
Fraser, G., Arcuri, A.: Whole test suite generation. IEEE Trans. Software Eng. 39(2), 276–291 (2013)
Fraser, G., Arcuri, A.: A large-scale evaluation of automated unit test generation using EvoSuite. ACM Trans. Softw. Eng. Methodol. 24(2), 81–842 (2014)
Fraser, G., Arcuri, A.: 1600 faults in 100 projects: automatically finding faults while achieving high coverage with EvoSuite. Empirical Softw. Eng. 20(3), 611–639 (2015)
Fraser, G., Arcuri, A., McMinn, P.: Test suite generation with memetic algorithms. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, GECCO 2013, pp. 1437–1444. ACM (2013)
Friedrich, T., Kötzing, T., Wagner, M.: A generic bet-and-run strategy for speeding up stochastic local search. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp. 801–807. AAAI Press (2017)
Kempka, J., McMinn, P., Sudholt, D.: Design and analysis of different alternating variable searches for search-based software testing. Theoret. Comput. Sci. 605, 1–20 (2015)
Lourenço, H.R., Martin, O.C., Stützle, T.: Iterated local search: framework and applications. In: Gendreau, M., Potvin, J.Y. (eds.) Handbook of Metaheuristics, pp. 363–397. Springer, Boston (2010). https://doi.org/10.1007/978-1-4419-1665-5_12
Martí, R.: Multi-start methods. In: Glover, F., Kochenberger, G.A. (eds.) Handbook of Metaheuristics, pp. 355–368. Springer, Boston (2003). https://doi.org/10.1007/0-306-48056-5_12
Mathesen, L., Yaghoubi, S., Pedrielli, G., Fainekos, G.: Falsification of cyber-physical systems with robustness uncertainty quantification through stochastic optimization with adaptive restart. In: 15th International Conference on Automation Science and Engineering, CASE, pp. 991–997. IEEE (2019)
McCabe, T.J.: A complexity measure. IEEE Trans. Software Eng. 2(4), 308–320 (1976)
McMinn, P.: Search-based software test data generation: a survey. Softw. Testing Verif. Reliab. 14(2), 105–156 (2004)
Orso, A., Rothermel, G.: Software testing: a research travelogue (2000–2014). In: Future of Software Engineering Proceedings, FOSE 2014, pp. 117–132. ACM (2014)
Panichella, A., Kifetew, F.M., Tonella, P.: Automated test case generation as a many-objective optimisation problem with dynamic selection of the targets. IEEE Trans. Software Eng. 44(2), 122–158 (2018)
Rojas, J.M., Vivanti, M., Arcuri, A., Fraser, G.: A detailed investigation of the effectiveness of whole test suite generation. Empirical Softw. Eng. 22(2), 852–893 (2017)
Shamshiri, S., Rojas, J.M., Fraser, G., McMinn, P.: Random or genetic algorithm search for object-oriented test suite generation? In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO, pp. 1367–1374. ACM (2015)
Tonella, P.: Evolutionary testing of classes. In: Avrunin, G.S., Rothermel, G. (eds.) Proceedings of the International Symposium on Software Testing and Analysis, ISSTA, pp. 119–128. ACM (2004)
Vogel, T., Tran, C., Grunske, L.: Does diversity improve the test suite generation for mobile applications? In: Nejati, S., Gay, G. (eds.) SSBSE 2019. LNCS, vol. 11664, pp. 58–74. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27455-9_5
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Müller, S., Vogel, T., Grunske, L. (2020). Bet and Run for Test Case Generation. In: Aleti, A., Panichella, A. (eds) Search-Based Software Engineering. SSBSE 2020. Lecture Notes in Computer Science(), vol 12420. Springer, Cham. https://doi.org/10.1007/978-3-030-59762-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-59762-7_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59761-0
Online ISBN: 978-3-030-59762-7
eBook Packages: Computer ScienceComputer Science (R0)