SpringerLink
Log in

Surrogate-assisted evolutionary sampling particle swarm optimization for high-dimensional expensive optimization

  • S.I. : Hybrid Approaches to Nature-inspired Optimization Algorithms and Their Applications
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Surrogate-assisted evolutionary algorithms have been widely employed for solving expensive optimization problems. To address high-dimensional expensive optimization problems, we propose an evolutionary sampling-assisted particle swarm optimization method, termed ESPSO. ESPSO consists of some evolutionary sampling-assisted strategies. It first improves the initialized population with some elite samples by evolutionary sampling. Secondly, during the optimization process, the method builds a local radial basis function model using the personal historical optimal data of the population to approximate the objective function landscape. Finally, surrogate-assisted local search and surrogate-assisted trust region search are designed to find promising candidate solutions for replacing individuals in the population to accelerate the search process. Behavioral research experiments of ESPSO verified these strategies have led to improvements in the search efficiency of the algorithm in various aspects, such as initialization, population update, and optimal solution promotion. We compared ESPSO with five state-of-the-art SAEAs using 18 benchmark functions, which show that ESPSO outperforms the other compared SAEAs and get the best average ranking of 2.194.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availibility

The data generated during and/or analyzed during the current study are available from the corresponding author upon reasonable request.

References

  1. Xue Y, Xue B, Zhang M (2019) Self-adaptive particle swarm optimization for large-scale feature selection in classification. ACM Trans Knowl Discov Data 13(5):1–27

    Article  Google Scholar 

  2. Wang F, Zhang H, Zhou A (2021) A particle swarm optimization algorithm for mixed-variable optimization problems. Swarm Evol Comput 60:100808

    Article  Google Scholar 

  3. Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359

    Article  MathSciNet  MATH  Google Scholar 

  4. Holland JH (1992) Genetic algorithms. Sci Am 267(1):66–73

    Article  Google Scholar 

  5. Ayyarao TS, Ramakrishna N, Elavarasan RM, Polumahanthi N, Rambabu M, Saini G, Khan B, Alatas B (2022) War strategy optimization algorithm: a new effective metaheuristic algorithm for global optimization. IEEE Access 10:25073–25105

    Article  Google Scholar 

  6. Xue Y, Wang Y, Liang J, Slowik A (2021) A self-adaptive mutation neural architecture search algorithm based on blocks. IEEE Comput Intell Mag 16(3):67–78

    Article  Google Scholar 

  7. Chugh T, Chakraborti N, Sindhya K, ** Y (2017) A data-driven surrogate-assisted evolutionary algorithm applied to a many-objective blast furnace optimization problem. Mater Manuf Process 32(10):1172–1178

    Article  Google Scholar 

  8. Manca AG, Pappalardo CM (2020) Topology optimization procedure of aircraft mechanical components based on computer-aided design, multibody dynamics, and finite element analysis. In: Design, simulation, manufacturing: the innovation exchange, Springer, pp 159–168

  9. Park J, Sandberg IW (1991) Universal approximation using radial-basis-function networks. Neural Comput 3(2):246–257

    Article  Google Scholar 

  10. Zhen H, Gong W, Wang L, Ming F, Liao Z (2021) Two-stage data-driven evolutionary optimization for high-dimensional expensive problems. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2021.3118783

    Article  Google Scholar 

  11. Zhen H, **ong S, Gong W, Wang L (2023) Neighborhood evolutionary sampling with dynamic repulsion for expensive multimodal optimization. Inf Sci 630:82–97. https://doi.org/10.1016/j.ins.2023.02.049

    Article  Google Scholar 

  12. Wang H, ** Y (2020) A random forest-assisted evolutionary algorithm for data-driven constrained multiobjective combinatorial optimization of trauma systems. IEEE Trans Cybern 50(2):536–549

    Article  Google Scholar 

  13. Kleijnen JP (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192(3):707–716

    Article  MathSciNet  MATH  Google Scholar 

  14. Poloczek J, Kramer O (2013) Local svm constraint surrogate models for self-adaptive evolution strategies. In: Timm IJ, Thimm M (eds) KI 2013: advances in artificial intelligence. Springer, Berlin, Heidelberg, pp 164–175

    Google Scholar 

  15. Liu B, Zhang Q, Gielen GG (2013) A Gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems. IEEE Trans Evol Comput 18(2):180–192

    Article  Google Scholar 

  16. Yi J, Shen Y, Shoemaker CA (2020) A multi-fidelity rbf surrogate-based optimization framework for computationally expensive multi-modal problems with application to capacity planning of manufacturing systems. Struct Multidiscip Optim 62(4):1787–1807

    Article  MathSciNet  Google Scholar 

  17. Nguyen BH, Xue B, Zhang M (2022) A constrained competitive swarm optimiser with an svm-based surrogate model for feature selection. IEEE Trans Evol Comput

  18. Wang H, ** Y, Doherty J (2017) Committee-based active learning for surrogate-assisted particle swarm optimization of expensive problems. IEEE Trans Cybern 47(9):2664–2677

    Article  Google Scholar 

  19. Zhen H, Gong W, Wang L (2022) Offline data-driven evolutionary optimization based on model selection. Swarm Evol Comput 71:101080

    Article  Google Scholar 

  20. Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492

    Article  MathSciNet  MATH  Google Scholar 

  21. Jiao R, Zeng S, Li C, Jiang Y, ** Y (2019) A complete expected improvement criterion for gaussian process assisted highly constrained expensive optimization. Inf Sci 471:80–96

    Article  MATH  Google Scholar 

  22. Emmerich MT, Giannakoglou KC, Naujoks B (2006) Single-and multiobjective evolutionary optimization assisted by Gaussian random field metamodels. IEEE Trans Evol Comput 10(4):421–439

    Article  Google Scholar 

  23. Zhang Q, Liu W, Tsang E, Virginas B (2009) Expensive multiobjective optimization by moea/d with Gaussian process model. IEEE Trans Evol Comput 14(3):456–474

    Article  Google Scholar 

  24. Xue Y, Tang Y, Xu X, Liang J, Neri F (2022) Multi-objective feature selection with missing data in classification. IEEE Trans Emerg Top Comput Intell 6(2):355–364

    Article  Google Scholar 

  25. Li F, Gao L, Garg A, Shen W, Huang S (2021) A comparative study of pre-screening strategies within a surrogate-assisted multi-objective algorithm framework for computationally expensive problems. Neural Comput Appl 33(9):4387–4416

    Article  Google Scholar 

  26. Liu Y, Liu J, Tan S, Yang Y, Li F (2022) A bagging-based surrogate-assisted evolutionary algorithm for expensive multi-objective optimization. Neural Comput Appl 34(14):12097–12118

    Article  Google Scholar 

  27. Zhen H, Gong W, Ling W (2021) Data-driven evolutionary sampling optimization for expensive problems. J Syst Eng Electron 32(2):318–330

    Article  Google Scholar 

  28. Wang X, Wang GG, Song B, Wang P, Wang Y (2019) A novel evolutionary sampling assisted optimization method for high-dimensional expensive problems. IEEE Trans Evol Comput 23(5):815–827

    Article  MathSciNet  Google Scholar 

  29. Tian J, Tan Y, Zeng J, Sun C, ** Y (2018) Multiobjective infill criterion driven gaussian process-assisted particle swarm optimization of high-dimensional expensive problems. IEEE Trans Evol Comput 23(3):459–472

    Article  Google Scholar 

  30. Yu H, Tan Y, Zeng J, Sun C, ** Y (2018) Surrogate-assisted hierarchical particle swarm optimization. Inf Sci 454:59–72

    Article  MathSciNet  Google Scholar 

  31. Cai X, Gao L, Li X (2020) Efficient generalized surrogate-assisted evolutionary algorithm for high-dimensional expensive problems. IEEE Trans Evol Comput 24(2):365–379

    Article  Google Scholar 

  32. Zhen H, Gong W, Wang L (2022) Evolutionary sampling agent for expensive problems. IEEE Trans Evol Comput. https://doi.org/10.1109/TEVC.2022.3177605

    Article  Google Scholar 

  33. Akyol S, Alatas B (2017) Plant intelligence based metaheuristic optimization algorithms. Artif Intell Rev 47:417–462

    Article  Google Scholar 

  34. Alatas B, Bingol H (2020) Comparative assessment of light-based intelligent search and optimization algorithms. Light Eng 28(6)

  35. Dong H, Dong Z (2020) Surrogate-assisted grey wolf optimization for high-dimensional, computationally expensive black-box problems. Swarm Evol Comput 57:100713

    Article  Google Scholar 

  36. Yu H, Kang L, Tan Y, Zeng J, Sun C (2021) A multi-model assisted differential evolution algorithm for computationally expensive optimization problems. Complex Intell Syst 7(5):2347–2371

    Article  Google Scholar 

  37. Nguyen HB, Xue B, Andreae P (2018) Pso with surrogate models for feature selection: static and dynamic clustering-based methods. Memet Comput 10(3):291–300

    Article  Google Scholar 

  38. Hao H, Zhou A, Qian H, Zhang H (2022) Expensive multiobjective optimization by relation learning and prediction. IEEE Trans Evol Comput 26:1157

    Article  Google Scholar 

  39. Pan L, He C, Tian Y, Wang H, Zhang X, ** Y (2018) A classification-based surrogate-assisted evolutionary algorithm for expensive many-objective optimization. IEEE Trans Evol Comput 23(1):74–88

    Article  Google Scholar 

  40. Chugh T, ** Y, Miettinen K, Hakanen J, Sindhya K (2016) A surrogate-assisted reference vector guided evolutionary algorithm for computationally expensive many-objective optimization. IEEE Trans Evol Comput 22(1):129–142

    Article  Google Scholar 

  41. Wang Y, Lin J, Liu J, Sun G, Pang T (2022) Surrogate-assisted differential evolution with region division for expensive optimization problems with discontinuous responses. IEEE Trans Evol Comput 26(4):780–792. https://doi.org/10.1109/TEVC.2021.3117990

    Article  Google Scholar 

  42. Tian J, Tan Y, Zeng J, Sun C, ** Y (2018) Multiobjective infill criterion driven gaussian process-assisted particle swarm optimization of high-dimensional expensive problems. IEEE Trans Evol Comput 23(3):459–472

    Article  Google Scholar 

  43. Cheng R, ** Y (2015) A social learning particle swarm optimization algorithm for scalable optimization. Inf Sci 291(C):43–60

    Article  MathSciNet  MATH  Google Scholar 

  44. Sun C, ** Y, Cheng R, Ding J, Zeng J (2017) Surrogate-assisted cooperative swarm optimization of high-dimensional expensive problems. IEEE Trans Evol Comput 21(4):644–660

    Article  Google Scholar 

  45. Li F, Cai X, Gao L, Shen W (2021) A surrogate-assisted multiswarm optimization algorithm for high-dimensional computationally expensive problems. IEEE Trans Cybern 51(3):1390–1402. https://doi.org/10.1109/TCYB.2020.2967553

    Article  Google Scholar 

  46. Chen G, Li Y, Zhang K, Xue X, Wang J, Luo Q, Yao C, Yao J (2021) Efficient hierarchical surrogate-assisted differential evolution for high-dimensional expensive optimization. Inf Sci 542:228–246

    Article  MathSciNet  MATH  Google Scholar 

  47. Meyer-Baese A, Schmid V (2014) Chapter 7 - foundations of neural networks. In: Meyer-Baese A, Schmid V (eds) Pattern recognition and signal analysis in medical imaging, 2nd edn. Academic Press, Oxford, pp 197–243

    Chapter  MATH  Google Scholar 

  48. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of ICNN’95-international Conference on Neural Networks, vol 4, pp 1942–1948 IEEE

  49. Wang F, Wang X, Sun S (2022) A reinforcement learning level-based particle swarm optimization algorithm for large-scale optimization. Inf Sci 602:298–312

    Article  Google Scholar 

  50. Bansal JC, Singh PK, Saraswat M, Verma A, Jadon SS, Abraham A (2011) Inertia weight strategies in particle swarm optimization. In: 2011 Third World Congress on Nature and Biologically Inspired Computing, pp 633–640 https://doi.org/10.1109/NaBIC.2011.6089659

  51. Helton JC, Davis FJ (2003) Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems. Reliabil Eng Syst Saf 81(1):23–69

    Article  Google Scholar 

  52. Suganthan PN, Hansen N, Liang JJ, Deb K, Tiwari S (2005) Problem definitions and evaluation criteria for the cec 2005 special session on real-parameter optimization. Nat Comput 341–357

  53. Wei F-F, Chen W-N, Yang Q, Deng J, Luo X-N, ** H, Zhang J (2021) A classifier-assisted level-based learning swarm optimizer for expensive optimization. IEEE Trans Evol Comput 25(2):219–233. https://doi.org/10.1109/TEVC.2020.3017865

    Article  Google Scholar 

  54. Chen C, Wang X, Dong H, Wang P (2022) Surrogate-assisted hierarchical learning water cycle algorithm for high-dimensional expensive optimization. Swarm Evol Comput 75:101169

    Article  Google Scholar 

Download references

Acknowledgements

This work was partly supported by the National Natural Science Foundation of China under Grant No. 62076225, and the Natural Science Foundation for Distinguished Young Scholars of Hubei under Grant No. 2019CFA081.

Author information

Authors and Affiliations

  1. College of System Engineering, National University of Defense Technology, Changsha, 410073, China

    Kuihua Huang & Rui Wang

  2. School of Computer Science, China University of Geosciences, Wuhan, 430074, China

    Huixiang Zhen & Wenyin Gong

  3. Equipment General Technology Laboratory, Bei**g Mechanical Equipment Research Institute, Bei**g, 100854, China

    Weiwei Bian

Authors
  1. Kuihua Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huixiang Zhen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenyin Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Weiwei Bian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenyin Gong.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, K., Zhen, H., Gong, W. et al. Surrogate-assisted evolutionary sampling particle swarm optimization for high-dimensional expensive optimization. Neural Comput & Applic (2023). https://doi.org/10.1007/s00521-023-08661-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00521-023-08661-3

Keywords

Search

Navigation