SpringerLink
Log in

Convolutional encoder–decoder network using transfer learning for topology optimization

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

State-of-the-art deep neural networks have achieved great success as an alternative to topology optimization by eliminating the iterative framework of the optimization process. However, models with strong predicting capabilities require massive data, which can be time-consuming, particularly for high-resolution structures. Transfer learning from pre-trained networks has shown promise in enhancing network performance on new tasks with a smaller amount of data. In this study, a U-net-based deep convolutional encoder–decoder network was developed for predicting high-resolution (256 × 256) optimized structures using transfer learning and fine-tuning for topology optimization. Initially, the VGG16 network pre-trained on ImageNet was employed as the encoder for transfer learning. Subsequently, the decoder was constructed from scratch and the network was trained in two steps. Finally, the results of models employing transfer learning and those trained entirely from scratch were compared across various core parameters, including different initial input iterations, fine-tuning epoch numbers, and dataset sizes. Our findings demonstrate that the utilization of transfer learning from the ImageNet pre-trained VGG16 network as the encoder can improve the final predicting performance and alleviate structural discontinuity issues in some cases while reducing training time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data and material availability

The datasets generated during and/or analyzed during the current study are available at https://github.com/gorkemcanates/Transfer-Learning-for-TO. All necessary information to obtain the topology optimization data and the neural network models that allow to reproduce the results was provided in this paper. The results described in this paper can be replicated by implementing this information.

Code availability

An open-source 2D topology optimization MATLAB code [3] was used to generate data. All codes for neural networks were written in Python 3.7. For neural networks, Pytorch library [33] was used. Our codes are available at https://github.com/gorkemcanates/Transfer-Learning-for-TO

Abbreviations

θ :

Angle of force

Β :

Constants in the optimizer

\(\mu\) :

Mean vector

\(\sigma\) :

Standard deviation vector

N c :

Fixed boundary condition

N f :

Single force application node

U :

Uniform distribution

v f :

Volume fraction

X min :

Maximum of x

X min :

Minimum of x

x n :

Normalized x vector

\({\widehat{y}}_{i}\) :

Predicted value

\({y}_{i}\) :

Target value

BCE:

Binary cross-entropy

CNN:

Convolutional neural network

DBN:

Deep belief network

FN:

False-negative

FP:

False positive

GAN:

Generative adversarial network

GPU:

Graphic processing unit

MAE:

Mean absolute error

MAPE:

Mean absolute percentage error

MSE:

Mean square error

ReLu:

Rectified linear unit

TN:

True negative

TP:

True positive

References

  1. Abdalla A, Cen H, Wan L et al (2019) Fine-tuning convolutional neural network with transfer learning for semantic segmentation of ground-level oilseed rape images in a field with high weed pressure. Comput Electron Agric 167:105091. https://doi.org/10.1016/j.compag.2019.105091

    Article  Google Scholar 

  2. Abueidda DW, Koric S, Sobh NA (2020) Topology optimization of 2D structures with nonlinearities using deep learning. Comput Struct 237:106283. https://doi.org/10.1016/j.compstruc.2020.106283

    Article  Google Scholar 

  3. Andreassen E, Clausen A, Schevenels M et al (2011) Efficient topology optimization in MATLAB using 88 lines of code. Struct Multidiscip Optim 43:1–16. https://doi.org/10.1007/s00158-010-0594-7

    Article  Google Scholar 

  4. Ates GC, Gorguluarslan RM (2021) Two-stage convolutional encoder-decoder network to improve the performance and reliability of deep learning models for topology optimization. Struct Multidiscip Optim 63:1927–1950. https://doi.org/10.1007/s00158-020-02788-w

    Article  MathSciNet  Google Scholar 

  5. Banga S, Gehani H, Bhilare S, et al. (2018) 3D Topology optimization using convolutional neural networks. ar**v preprint ar**v:180807440

  6. Cang R, Xu Y, Chen S et al (2017) Microstructure representation and reconstruction of heterogeneous materials via deep belief network for computational material design. J Mech Des 139:071404. https://doi.org/10.1115/1.4036649

    Article  Google Scholar 

  7. Cui B, Chen X, Lu Y (2020) Semantic segmentation of remote sensing images using transfer learning and deep convolutional neural network with dense connection. IEEE Access 8:116744–116755. https://doi.org/10.1109/ACCESS.2020.3003914

    Article  Google Scholar 

  8. Doi S, Sasaki H, Igarashi H (2019) Multi-objective topology optimization of rotating machines using deep learning. IEEE Trans Magn 55:7202605. https://doi.org/10.1109/TMAG.2019.2899934

    Article  Google Scholar 

  9. Frid-Adar M, Ben-Cohen A, Amer R, Greenspan H (2018) Improving the segmentation of anatomical structures in chest radiographs using U-net with an imagenet pre-trained encoder. ar**v 1810.02113

  10. Goodfellow IJ, Pouget-Abadie J, Mirza M, et al (2014) Generative adversarial nets. Adv Neural Inf Process Syst. pp 2672–2680

  11. Gorguluarslan RM, Ates GC (2023) Evaluation of deep learning networks for predicting truss topology optimization results. ASME International Mechanical Engineering Congress and Exposition, Proceedings (IMECE) https://doi.org/10.1115/IMECE2022-95870

  12. He K, Girshick R, Dollar P (2019) Rethinking imageNet pre-training. In: Proceedings of the IEEE International Conference on Computer Vision. pp 4917–4926

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp 770–778

  14. Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554. https://doi.org/10.1162/neco.2006.18.7.1527

    Article  MathSciNet  PubMed  Google Scholar 

  15. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. pp 2261–2269

  16. Huh M, Agrawal P, Efros AA (2016) What makes ImageNet good for transfer learning? ar**v 1608.08614

  17. Iglovikov V, Seferbekov S, Buslaev A, Shvets A (2018) TernausNetV2: Fully convolutional network for instance segmentation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. pp 228–232

  18. Iglovikov V, Shvets A (2018) TernausNet: U-Net with VGG11 encoder pre-trained on imagenet for image segmentation. ar**v 1801.05746

  19. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: 32nd International Conference on Machine Learning, ICML 2015. pp 448–456

  20. Jiang H, Nie Z, Yeo R, et al (2020) Stressgan: a generative deep learning model for 2D stress distribution prediction. In: Proceedings of the ASME Design Engineering Technical Conference. Virtual, Online, p V11BT11A023

  21. Kallioras NA, Kazakis G, Lagaros ND (2020) Accelerated topology optimization by means of deep learning. Struct Multidiscip Optim 62:1185–1212. https://doi.org/10.1007/s00158-020-02545-z

    Article  MathSciNet  Google Scholar 

  22. Kallioras NA, Lagaros ND (2020) DL-Scale: Deep learning for model upgrading in topology optimization. In: Procedia Manufacturing. pp 433–440

  23. Kingma DP, Ba JL (2015) Adam: A method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. p ar**v:1412.6980

  24. Li B, Huang C, Li X et al (2019) Non-iterative structural topology optimization using deep learning. Comput Aided Des 115:172–180. https://doi.org/10.1016/j.cad.2019.05.038

    Article  Google Scholar 

  25. Lin Q, Hong J, Liu Z et al (2018) Investigation into the topology optimization for conductive heat transfer based on deep learning approach. Int Commun Heat Mass Transfer 97:103–109. https://doi.org/10.1016/j.icheatmasstransfer.2018.07.001

    Article  Google Scholar 

  26. Lin Q, Liu Z, Hong J (2019) Method for directly and instantaneously predicting conductive heat transfer topologies by using supervised deep learning. Int Commun Heat Mass Transfer 109:104368. https://doi.org/10.1016/j.icheatmasstransfer.2019.104368

    Article  Google Scholar 

  27. Liu X, Deng Z, Yang Y (2019) Recent progress in semantic image segmentation. Artif Intell Rev. https://doi.org/10.1007/s10462-018-9641-3

    Article  Google Scholar 

  28. Nair V, Hinton GE (2010) Rectified linear units improve Restricted Boltzmann machines. In: ICML 2010 - Proceedings, 27th International Conference on Machine Learning. pp 807–814

  29. Nakamura K, Suzuki Y (2020) Deep learning-based topological optimization for representing a user-specified design area. ar**v 2004.05461

  30. Nie Z, Lin T, Jiang H, Kara LB (2020) Topologygan: topology optimization using generative adversarial networks based on physical fields over the initial domain. J Mech Des 143:031715. https://doi.org/10.1115/1.4049533

    Article  Google Scholar 

  31. Oh S, Jung Y, Kim S et al (2019) Deep generative design: integration of topology optimization and generative models. J Mech Des 141:111405. https://doi.org/10.1115/1.4044229

    Article  Google Scholar 

  32. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359. https://doi.org/10.1109/TKDE.2009.191

    Article  Google Scholar 

  33. Paszke A, Gross S, Massa F et al (2019) PyTorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8024–8035

    Google Scholar 

  34. Peng C, **ao T, Li Z, et al (2018) MegDet: A large mini-batch object detector. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp 6181–6189

  35. Pondenkandath V, Alberti M, Eichenberger N, et al (2018) Identifying cross-depicted historical motifs. Proceedings of International Conference on Frontiers in Handwriting Recognition, ICFHR 2018-August:333–338. https://doi.org/10.1109/ICFHR-2018.2018.00065

  36. Ramu P, Thananjayan P, Acar E et al (2022) A survey of machine learning techniques in structural and multidisciplinary optimization. Struct Multidiscip Optimiz 65:1–31. https://doi.org/10.1007/S00158-022-03369-9

    Article  Google Scholar 

  37. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, pp 234–241

    Google Scholar 

  38. Russakovsky O, Deng J, Su H et al (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115:211–252. https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  39. Sasaki H, Igarashi H (2019) Topology optimization accelerated by deep learning. IEEE Trans Magn 55:7401305. https://doi.org/10.1109/TMAG.2019.2901906

    Article  Google Scholar 

  40. Sharpe C, Wiest T, Wang P, Seepersad CC (2019) A comparative evaluation of supervised machine learning classification techniques for engineering design applications. J Mech Des 141:121404. https://doi.org/10.1115/1.4044524

    Article  Google Scholar 

  41. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. San Diego, CA, USA, p ar**v:1409.1556

  42. Singh MS, Pondenkandath V, Zhou B, et al (2017) Transforming sensor data to the image domain for deep learning—An application to footstep detection. Proceedings of the International Joint Conference on Neural Networks 2017-May:2665–2672. https://doi.org/10.1109/IJCNN.2017.7966182

  43. Sosnovik I, Oseledets I (2019) Neural networks for topology optimization. Russ J Numer Anal Math Model 34:215–223. https://doi.org/10.1515/rnam-2019-0018

    Article  MathSciNet  Google Scholar 

  44. Studer L, Alberti M, Pondenkandath V, et al (2019) A comprehensive study of imagenet pre-training for historical document image analysis. Proceedings of the International Conference on Document Analysis and Recognition, ICDAR 720–725. https://doi.org/10.1109/ICDAR.2019.00120

  45. Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E (2018) Deep learning for computer vision: a brief review. Comput Intell Neurosci 2018

  46. Wang L, Shi D, Zhang B et al (2023) Deep learning driven real time topology optimization based on improved convolutional block attention (Cba-U-Net) model. Eng Anal Bound Elem 147:112–124. https://doi.org/10.1016/J.ENGANABOUND.2022.11.034

    Article  MathSciNet  CAS  Google Scholar 

  47. Wu Y, Bai Y, Lan Z, Yao S (2023) A structural-similarity conditional gan method to generate real-time topology for shell-infill structures. Int J Comput Methods. https://doi.org/10.1142/S0219876223410074

    Article  Google Scholar 

  48. Wu Y, He K (2020) Group normalization. Int J Comput Vis 128:742–755. https://doi.org/10.1007/s11263-019-01198-w

    Article  Google Scholar 

  49. Wurm M, Stark T, Zhu XX et al (2019) Semantic segmentation of slums in satellite images using transfer learning on fully convolutional neural networks. ISPRS J Photogramm Remote Sens 150:59–69. https://doi.org/10.1016/j.isprsjprs.2019.02.006

    Article  ADS  Google Scholar 

  50. **ang C, Wang D, Pan Y et al (2022) Accelerated topology optimization design of 3D structures based on deep learning. Struct Multidiscip Optim 65:1–18. https://doi.org/10.1007/S00158-022-03194-0/TABLES/7

    Article  Google Scholar 

  51. Yang Z, Li X, Brinson LC et al (2018) Microstructural materials design via deep adversarial learning methodology. J Mech Des 140:111416. https://doi.org/10.1115/1.4041371

    Article  Google Scholar 

  52. Yao Z, Cao Y, Zheng S, et al (2021) Cross-iteration batch normalization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

  53. Yu Y, Hur T, Jung J, Jang IG (2019) Deep learning for determining a near-optimal topological design without any iteration. Struct Multidiscip Optim 59:787–799. https://doi.org/10.1007/s00158-018-2101-5

    Article  Google Scholar 

  54. Zhang Y, Chen A, Peng B, et al (2019) A deep Convolutional Neural Network for topology optimization with strong generalization ability. ar**v preprint ar**v:190107761

  55. Zhuang F, Qi Z, Duan K et al (2021) A comprehensive survey on transfer learning. Proc IEEE 109:43–76

    Article  Google Scholar 

Download references

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, TOBB University of Economics and Technology, Sogutozu Cd. No: 43 Cankaya, 06560, Ankara, Turkey

    Gorkem Can Ates & Recep M. Gorguluarslan

Authors
  1. Gorkem Can Ates
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Recep M. Gorguluarslan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Recep M. Gorguluarslan.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ates, G.C., Gorguluarslan, R.M. Convolutional encoder–decoder network using transfer learning for topology optimization. Neural Comput & Applic 36, 4435–4450 (2024). https://doi.org/10.1007/s00521-023-09308-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09308-z

Keywords

Search

Navigation