Abstract
Automatic programming is an efficient technique that has contributed to an important development in the artificial intelligence and machine learning fields. In this chapter, we introduce the technique called Variable Neighborhood Programming (VNP) that was inspired by the principle of the Variable Neighborhood Search (VNS) algorithm. VNP starts from a single solution presented by a program, and the search for a good quality global solution (program) continues by exploring different neighborhoods. The goal of our algorithm is to generate a good representative program adequate to a selected problem. VNP takes the advantages of the systematic change of neighborhood structures randomly or within a local search algorithm to diversify or intensify search through the solution space. To show its efficiency and usefulness, the VNP method is applied first for solving the symbolic regression problem (VNP-SRP) and tested and compared on usual test instances from the literature. In addition, the VNP-SRP method is tested in finding formulas for life expectancy as a function of some health care economic factors in 18 Russian districts. Finally, the VNP is implemented on prediction and classification problems and tested on real-life maintenance railway problems from the US railway system.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Andersson, M.: Strategic planning of track maintenance – state of the art. Technical Report 02-035 (2002)
Andrade, A.R., Teixeira, P.F.: Hierarchical Bayesian modelling of rail track geometry degradation. Proc. Inst. Mech. Eng. F J. Rail Rapid Transit 227(4), 364–375 (2013)
Andrade, A., Teixeira, P.: Statistical modelling of railway track geometry degradation using Hierarchical Bayesian models. Reliab. Eng. Syst. Saf. 142, 169–183 (2015)
Arnaldo, I., Krawiec, K., O’Reilly, U.-M.: Multiple regression genetic programming. In: Proceedings of the 2014 Conference on Genetic and Evolutionary Computation – GECCO ’14, pp. 879–886. ACM Press, New York (2014)
Association of American Railroads (AAR). https://www.aar.org/todays-railroads (2015)
Bouaziz, S., Dhahri, H., Alimi, A.M., Abraham, A.: A hybrid learning algorithm for evolving Flexible Beta Basis Function Neural Tree Model. Neurocomputing 117, 107–117 (2013)
Brimberg, J., Mladenović, N., Todosijević, R., Urošević, D.: Less is more: solving the Max-Mean diversity problem with variable neighborhood search. Inf. Sci. 382–383, 179–200 (2017)
Brown, B.M., Chen, S.X.: Beta-Bernstein smoothing for regression curves with compact support. Scand. J. Stat. 26(1), 47–59 (1999)
Cai, W., Pacheco-Vega, A., Sen, M., Yang, K.: Heat transfer correlations by symbolic regression. Int. J. Heat Mass Trans. 49(23–24), 4352–4359 (2006)
Cannon, D.F., Edel, K.-O., Grassie, S.L., Sawley, K.: Rail defects: an overview. Fatigue Fract. Eng. Mater. Struct. 26(10), 865–886 (2003)
Castelli, M., Vanneschi, L., Silva, S.: Prediction of the unified Parkinson’s disease rating scale assessment using a genetic programming system with geometric semantic genetic operators. Expert Syst. Appl. 41(10), 4608–4616 (2014)
Castelli, M., Trujillo, L., Vanneschi, L.: Energy consumption forecasting using semantic-based genetic programming with local search optimizer. Comput. Intel. Neurosci. 2015, 971908 (2015)
Choi, W.-J., Choi, T.-S.: Genetic programming-based feature transform and classification for the automatic detection of pulmonary nodules on computed tomography images. Inf. Sci. 212, 57–78 (2012)
Gonçalves-de-Silva, K., Aloise, D., Xavier-de-Souza, S., Mladenovic, N.: Less is more: simplified Nelder-Mead method for large unconstrained optimization. Yugosl. J. Oper. Res. 28, 153–169 (2018)
Costa, L.R., Aloise, D., Mladenović, N.: Less is more: basic variable neighborhood search heuristic for balanced minimum sum-of-squares clustering. Inf. Sci. 415-416, 247–253 (2017)
de Arruda Pereira, M., Davis Júnior, C.A., Gontijo Carrano, E., de Vasconcelos, J.A.A.: A niching genetic programming-based multi-objective algorithm for hybrid data classification. Neurocomputing 133, 342–357 (2014)
De Boor, C.: A Practical Guide to Splines: With 32 Figures. Springer, Berlin (2001)
Deklel, A.K., Saleh, M.A., Hamdy, A.M., Saad, E.M.: Transfer learning with long term artificial neural network memory (LTANN-MEM) and neural symbolization algorithm (NSA) for solving high dimensional multi-objective symbolic regression problems. In: 2017 34th National Radio Science Conference (NRSC), pp. 343–352. IEEE, Piscataway (2017)
Elleuch, S., Jarboui, B., Mladenovic, N.: Variable neighborhood programming – a new automatic programming method in artificial intelligence. Technical report, G-2016-92, GERAD, Montreal (2016)
Elleuch, S., Hansen, P., Jarboui, B., Mladenović, N.: New VNP for automatic programming. Elect. Notes Discrete Math. 58, 191–198 (2017)
Fernandez de Canete, J., Del Saz-Orozco, P., Baratti, R., Mulas, M., Ruano, A., Garcia-Cerezo, A.: Soft-sensing estimation of plant effluent concentrations in a biological wastewater treatment plant using an optimal neural network. Expert Syst. Appl. 63, 8–19 (2016)
Friedman, J. H.: Multivariate adaptive regression splines. Annal. Stat. 19(1), 1–67 (1991)
Galar, M., Fernández, A., Barrenechea, E., Bustince, H., Herrera, F.: An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes. Patt. Recog. 44(8), 1761–1776 (2011)
GarcÃa-Torres, M., Gómez-Vela, F., Melián-Batista, B., Moreno-Vega, J.M.: High-dimensional feature selection via feature grou**: a variable neighborhood search approach. Inf. Sci. 326, 102–118 (2016)
Ghaddar, B., Sakr, N., Asiedu, Y.: Spare parts stocking analysis using genetic programming. Europ. J. Oper. Res. 252(1), 136–144 (2016)
Graupe, D.: Principles of Artificial Neural Networks. World Scientific, Singapore (2007)
Guler, H.: Prediction of railway track geometry deterioration using artificial neural networks: a case study for Turkish state railways. Struct. Infrastruct. Eng. 10(5), 614–626 (2014)
Gustavsson, E., Patriksson, M., Strömberg, A.-B., Wojciechowski, A., Önnheim, M.: Preventive maintenance scheduling of multi-component systems with interval costs. Comput. Ind. Eng. 76, 390–400 (2014)
Hansen, P., Mladenović, N.: Variable neighborhood search. In: Search Methodologies, pp. 211–238. Springer, Boston (2005)
Hansen, P., Mladenović, N., Pérez, JAM.: Variable neighbourhood search: methods and applications. Ann. Oper. Res. 175(1), 367–407
He, Q., Li, H., Bhattacharjya, D., Parikh, D.P., Hampapur, A.: Track geometry defect rectification based on track deterioration modelling and derailment risk assessment. J. Oper. Res. Soc. 66(3), 392–404 (2015)
Healthcare in Russia. Stat. book./Rosstat (2006)
Healthcare in Russia. Stat. book./Rosstat (2007)
Healthcare in Russia. Stat. book./Rosstat (2009)
Healthcare in Russia. Stat. book./Rosstat (2011)
Healthcare in Russia. Stat. book./Rosstat (2015)
Healthcare in Russia. Stat. book./Rosstat (2017)
Health at a Glance 2017: OECD indicators. http://dx.doi.org/10.1787/888933602215
Health at a Glance 2017: OECD indicators. http://dx.doi.org/10.1787/888933602272
Health care: current status and possible development scenarios. In: Dokl. to the 18th April International Scientific conference on the Problems of Economic and Social Development, Moscow, April 11–14, 2017. House of the Higher School of Economics (2017)
Hoai, N., McKay, R., Essam, D., Chau, R.: Solving the symbolic regression problem with tree-adjunct grammar guided genetic programming: the comparative results. In: Proceedings of the 2002 Congress on Evolutionary Computation. CEC’02 (Cat. No. 02TH8600), vol. 2, pp. 1326–1331. IEEE, Piscataway (2002)
Hoang, T.-H., Essam, D., McKay, B., Hoai, N.-X.: Building on success in genetic programming: adaptive variation and developmental evaluation. In: Advances in Computation and Intelligence, pp. 137–146. Springer, Berlin (2007)
Howard, D., Roberts, S., Brankin, R.: Target detection in SAR imagery by genetic programming. Adv. Eng. Softw. 30(5), 303–311 (1999)
Icke, I., Bongard, J.C.: Improving genetic programming based symbolic regression using deterministic machine learning. In: 2013 IEEE Congress on Evolutionary Computation, pp. 1763–1770. IEEE, Piscataway (2013)
Jaba, E., Balan, C.B., Robu, I.-B.: The relationship between life expectancy at birth and health expenditures estimated by a cross-country and time-series analysis. Proc. Eco. Finance 15, 108–114 (2014). Emerging Markets Queries in Finance and Business (EMQ 2013).
Jiaqiu, W., Ioannis, T., Chen, Z.: A space–time delay neural network model for travel time prediction. Eng. Appl. Artif. Intell. 52, 145–160 (2016)
Johnson, C.G.: Genetic Programming Crossover: Does It Cross over? pp. 97–108. Springer, Berlin (2009)
Kantardzic, M.: Data Mining Concepts, Models, Methods, and Algorithms. Wiley-IEEE Press, Hoboken (2011)
Karaboga, D., Ozturk, C., Karaboga, N., Gorkemli, B.: Artificial bee colony programming for symbolic regression. Inf. Sci. 209, 1–15 (2012)
Keijzer, M.: Improving Symbolic Regression with Interval Arithmetic and Linear Scaling, pp. 70–82. Springer, Berlin (2003)
Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)
Koza, J.R.: Genetic Programming II: Automatic Discovery of Reusable Programs. MIT Press, Cambridge (1994)
Kristjanpoller, W., Minutolo, M.C.: Forecasting volatility of oil price using an artificial neural network-GARCH model. Expert Syst. Appl. 65, 233–241 (2016)
Lalonde, M.: A new perspective on the health of Canadians. Technical report (1994)
Lamson, S.T., Hastings, N.A.J., Willis, R.J.: Minimum cost maintenance in heavy haul rail track. J. Oper. Res. Soc. 34(3), 211 (1983)
Lane, F., Azad, R., Ryan, C.: On effective and inexpensive local search techniques in genetic programming regression. In: Parallel Problem Solving from Nature – PPSN XIII, vol. 8672. Lecture Notes in Computer Science. Springer International Publishing, Berlin (2014)
Lidén, T.: Railway infrastructure maintenance – a survey of planning problems and conducted research. Trans. Res. Proc. 10, 574–583 (2015)
Life expectancy increasing in Russia, experts claim. https://www.vedomosti.ru/economics/articles/2018/05/29/770996-rosta-prodolzhitelnosti-zhizni
Lubitz, J., Cai, L., Kramarow, E., Lentzner, H.: Health, life expectancy, and health care spending among the elderly. N. Engl. J. Med. 349(11), 1048–1055 (2003). PMID: 12968089
Ly, D.L., Lipson, H.: Learning symbolic representations of hybrid dynamical systems. J. Mach. Learn. Res. 13(Dec), 3585–3618 (2012)
Macchi, M., Garetti, M., Centrone, D., Fumagalli, L., Piero Pavirani, G.: Maintenance management of railway infrastructures based on reliability analysis. Reliab. Eng. Syst. Saf. 104, 71–83 (2012)
Mladenović, N., Hansen, P.: Variable neighborhood search. Comput. Oper. Res. 24(11), 1097–1100 (1997)
Mladenović, N., Urošević, D.: Variable Neighborhood Search for the K-Cardinality Tree. Metaheuristics: Computer Decision-Making, Applied Optimization. Springer, Boston (2003)
Mladenović, N., Todosijević, R., Urošević, D.: Less is more: basic variable neighborhood search for minimum differential dispersion problem. Inf. Sci. 326, 160–171 (2016)
Mladenović, M., Delot, T., Laporte, G., Wilbaut, C.: The parking allocation problem for connected vehicles. J. Heuristics 26, 377–399 (2020)
Mladenović, N., Alkandari, A., Pei, J., Todosijević, R., Pardalos, P.M.: Less is more approach: basic variable neighborhood search for the obnoxious p-median problem. Int. Trans. Oper. Res. 27(1), 480–493 (2020)
Muggleton, S., de Raedt, L.: Inductive logic programming: theory and methods. J. Logic Program. 19–20, 629–679 (1994)
Musilek, P., Lau, A., Reformat, M., Wyardscott, L.: Immune programming. Inf. Sci. 176(8), 972–1002 (2006)
Nguyen, S., Zhang, M., Member, S., Johnston, M., Tan, K.C.: Automatic programming via iterated local search for dynamic job shop scheduling. IEEE Trans. Cybern. 45(1), 1–14 (2015)
O’Neill, M., Brabazon, A.: Grammatical swarm. In: Genetic and Evolutionary Computation Conference (GECCO), pp. 163–174. Springer, Berlin (2004)
Pei, J., Mladenović, N., Urošević, D., Brimberg, J., Liu, X.: Solving the traveling repairman problem with profits: a novel variable neighborhood search approach. Inf. Sci. 507, 108–123 (2020)
Peng, F., Kang, S., Li, X., Ouyang, Y., Somani, K., Acharya, D.: A heuristic approach to the railroad track maintenance scheduling problem. Comput. Aided Civ. Inf. Eng. 26(2), 129–145 (2011)
Peng, F., Ouyang, Y., Somani, K.: Optimal routing and scheduling of periodic inspections in large-scale railroad networks. J. Rail Transp. Plann. Manage. 3(4), 163–171 (2013)
Peng, Y., Yuan, C., Qin, X., Huang, J., Shi, Y.: An improved gene expression programming approach for symbolic regression problems. Neurocomputing 137, 293–301 (2014)
Rad, H.I., Feng, J., Iba, H.: GP-RVM: genetic programming-based symbolic regression using relevance vector machine. Ar**v: 1806,02502v (2018)
Roux, O., Cyril, F.: Ant programming: or how to use ants for automatic programming. In: International Conference on Swarm Intelligence, pp. 121–129 (2000)
Russian statistical yearbook. Rosstat (2018)
Shcherbakova, E.: Life expectancy and health care in OECD countries. Technical Report, Demoscope weekly (2018)
Stadtmüller, U.: Asymptotic properties of nonparametric curve estimates. Period. Math. Hung. 17(2), 83–108 (1986)
Uy, N.Q., Hoai, N.X., O’Neill, M., McKay, R.I., Galván-López, E.: Semantically-based crossover in genetic programming: application to real-valued symbolic regression. Genet. Program Evolvable Mach. 12(2), 91–119 (2011)
Witten, I.H., Frank, E., Hall, M.A.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, Burlington (2011)
Wolpert, D., Macready, W.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)
Wong, P., Zhang, M.: SCHEME: caching subtrees in genetic programming. In: 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence), pp. 2678–2685. IEEE, Piscataway (2008)
World health statistics 2017: monitoring health for the SDGs, Sustainable Development Goals. Technical Report, World Health Organization (2017)
Yao, X., Liu, Y., Lin, G.: Evolutionary programming made faster. IEEE Trans. Evol. Comput. 3(2), 82–102 (1999)
Yousefikia, M., Moridpour, S., Setunge, S., Mazloumi, E.: Modeling degradation of tracks for maintenance planning on a tram line. J. Traffic Logist. Eng. 2(2), 86–91 (2014)
Zhao, J., Chan, A.H.C., Stirling, A.B., Madelin, K.B.: Optimizing policies of railway ballast tam** and renewal. Trans. Res. Record J. Trans. Res. Board 1943(1), 50–56 (2006)
Zhao, J., Chan, A.H.C., Burrow, M.P.N.: Reliability analysis and maintenance decision for railway sleepers using track condition information. J. Oper. Res. Soc. 58(8), 1047–1055 (2007)
Acknowledgements
This publication is partially supported by the Khalifa University of Science and Technology under Award No. RC2 DSO. This research is also partially supported by the framework of Grant BR05236839, development of information technologies and systems for stimulation of personality’s sustainable development as one of the bases of development of digital Kazakhstan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Mladenovic, N., Jarboui, B., Elleuch, S., Mussabayev, R., Rusetskaya, O. (2021). Variable Neighborhood Programming as a Tool of Machine Learning. In: Pardalos, P.M., Rasskazova, V., Vrahatis, M.N. (eds) Black Box Optimization, Machine Learning, and No-Free Lunch Theorems. Springer Optimization and Its Applications, vol 170. Springer, Cham. https://doi.org/10.1007/978-3-030-66515-9_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-66515-9_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66514-2
Online ISBN: 978-3-030-66515-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)