Abstract
Hyper-heuristics have risen as a recurrent method to solve combinatorial optimization problems since they use a set of heuristics selectively according to the problem state. Although many ideas have been developed to produce hyper-heuristics, a recent trend involves treating the heuristic selection problem as a classification one. This allows the introduction of machine learning elements into the hyper-heuristic process. This work explores creating hyper-heuristics using Machine Learning classifiers to solve the Knapsack Problem, a fascinating and well-studied combinatorial problem. We propose two approaches to these hyper-heuristics: a dynamical approach, where the hyper-heuristic may change heuristics throughout the whole solving process, and a static approach, where the hyper-heuristic makes one initial choice of heuristic and no further changes are allowed. Our results confirm that hyper-heuristics powered by machine learning techniques can deal with the Knapsack problem and obtain competent results. Besides, we also observed a clear superiority in the performance of the hyper-heuristics running under the static approach concerning the dynamic counterpart.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
These instances are publicly available at https://bit.ly/3wvxPly.
References
Assi, M., Haraty, R.A.: A survey of the knapsack problem, pp. 1–6. IEEE, November 2018. https://doi.org/10.1109/ACIT.2018.8672677
Best, M.J., Ritter, K.: Linear Programming Active Set Analysis and Computer Programs. Prentice Hall Englewood Cliffs, N.J., January 1985
Bhargava, A.Y.: Dynamic Programming, vol. 1. Manning Publications, Shelter Island, 1 edn., May 2016
Bretthauer, K.M., Shetty, B.: The nonlinear knapsack problem - algorithms and applications. Eur. J. Oper. Res. 138, 459–472 (2002). https://doi.org/10.1016/S0377-2217(01)00179-5
Burke, E.K.: Hyper-heuristics: a survey of the state of the art. J. Oper. Res. Soc. 64(12), 1695–1724 (2013)
Cacchiani, V., Iori, M., Locatelli, A., Martello, S.: Knapsack problems - an overview of recent advances. part i: Single knapsack problems. Comput. Oper. Res. 143, 105692 (2022). https://doi.org/10.1016/j.cor.2021.105692, https://www.sciencedirect.com/science/article/pii/S0305054821003877
Drake, J.H., Hyde, M., Ibrahim, K., Ozcan, E.: A genetic programming hyper-heuristic for the multidimensional knapsack problem. Kybernetes 43, 1500–1511 (2014). https://doi.org/10.1108/K-09-2013-0201
Duhart, B., Camarena, F., Ortiz-Bayliss, J.C., Amaya, I., Terashima-Marín, H.: An experimental study on ant colony optimization hyper-heuristics for solving the knapsack problem. In: Martínez-Trinidad, J.F., Carrasco-Ochoa, J.A., Olvera-López, J.A., Sarkar, S. (eds.) MCPR 2018. LNCS, vol. 10880, pp. 62–71. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-92198-3_7
Eddy, S.R.: What is dynamic programming? Nat. Biotechnol. 22, 909–910 (2004). https://doi.org/10.1038/nbt0704-909
Žerovnik, J.: Heuristics for np-hard optimization problems - simpler is better!? Logist. Sustain. Transp. 6, 1–10 (2015). https://doi.org/10.1515/jlst-2015-0006
Kellerer, H., Pferschy, U., Pisinger, D.: Basic Algorithm Concepts, pp. 27–29. Springer, Berlin, 1 edn., October 2004. https://doi.org/10.1007/978-3-540-24777-7
Mischek, F., Musliu, N.: Reinforcement learning for cross-domain hyper-heuristics. In: Raedt, L.D. (ed.) Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22. pp. 4793–4799. International Joint Conferences on Artificial Intelligence Organization, July 2022. https://doi.org/10.24963/ijcai.2022/664, main Track
Mougouei, D., Powers, D.M.W., Moeini, A.: An Integer Linear Programming Model for Binary Knapsack Problem with Dependent Item Values, vol. 10400, pp. 144–154. Springer, Cham, 1 edn., July 2017. https://doi.org/10.1007/978-3-319-63004-5_12
Ochoa, G., et al.: HyFlex: a benchmark framework for cross-domain heuristic search. In: Hao, J.-K., Middendorf, M. (eds.) EvoCOP 2012. LNCS, vol. 7245, pp. 136–147. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29124-1_12
Olivas, F., Amaya, I., Ortiz-Bayliss, J.C., Conant-Pablos, S.E., Terashima-Marin, H.: A fuzzy hyper-heuristic approach for the 0-1 knapsack problem, pp. 1–8. IEEE, July 2020. https://doi.org/10.1109/CEC48606.2020.9185710
Ortiz-Bayliss, J.C., Amaya, I., Cruz-Duarte, J.M., Gutierrez-Rodriguez, A.E., Conant-Pablos, S.E., Terashima-Marín, H.: A general framework based on machine learning for algorithm selection in constraint satisfaction problems. Appl. Sci. 11(6) (2021). https://doi.org/10.3390/app11062749
Ortiz-Bayliss, J.C., Terashima-Marín, H., Conant-Pablos, S.E.: Neural networks to guide the selection of heuristics within constraint satisfaction problems. In: Martínez-Trinidad, J.F., Carrasco-Ochoa, J.A., Ben-Youssef Brants, C., Hancock, E.R. (eds.) MCPR 2011. LNCS, vol. 6718, pp. 250–259. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21587-2_27
Pillay, N., Beckedahl, D.: Evohyp - a java toolkit for evolutionary algorithm hyper-heuristics, pp. 2706–2713. IEEE, June 2017. https://doi.org/10.1109/CEC.2017.7969636
Plata-González, L.F., Amaya, I., Ortiz-Bayliss, J.C., Conant-Pablos, S.E., Terashima-Marín, H., Coello Coello, C.A.: Evolutionary-based tailoring of synthetic instances for the knapsack problem. Soft Comput. 23, 12711–12728 (2019). https://doi.org/10.1007/s00500-019-03822-w
RASHID, M.H.: A GPU accelerated parallel heuristic for the 2d knapsack problem with rectangular pieces, pp. 783–787. IEEE (11 2018). https://doi.org/10.1109/UEMCON.2018.8796818
Sánchez-Díaz, X., Ortiz-Bayliss, J.C., Amaya, I., Cruz-Duarte, J.M., Conant-Pablos, S.E., Terashima-Marín, H.: A feature-independent hyper-heuristic approach for solving the knapsack problem. Appl. Sci. 11, 10209 (2021). https://doi.org/10.3390/app112110209
Tu, C., Bai, R., Aickelin, U., Zhang, Y., Du, H.: A deep reinforcement learning hyper-heuristic with feature fusion for online packing problems. Expert Syst. Appl. 230 (2023). https://doi.org/10.1016/j.eswa.2023.120568
Tyasnurita, R., Özcan, E., John, R.I.: Learning heuristic selection using a time delay neural network for open vehicle routing. In: 2017 IEEE Congress on Evolutionary Computation (CEC), pp. 1474–1481 (2017). https://api.semanticscholar.org/CorpusID:5959987
Zeng, Z., **ong, C., Yuan, X., Bai, Y., **, Y., Lu, D., Lian, L.: Information-driven path planning for hybrid aerial underwater vehicles, April 2022
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zárate-Aranda, J.E., Ortiz-Bayliss, J.C. (2024). An Exploratory Study on Machine-Learning-Based Hyper-heuristics for the Knapsack Problem. In: Mezura-Montes, E., Acosta-Mesa, H.G., Carrasco-Ochoa, J.A., Martínez-Trinidad, J.F., Olvera-López, J.A. (eds) Pattern Recognition. MCPR 2024. Lecture Notes in Computer Science, vol 14755. Springer, Cham. https://doi.org/10.1007/978-3-031-62836-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-62836-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-62835-1
Online ISBN: 978-3-031-62836-8
eBook Packages: Computer ScienceComputer Science (R0)
