Abstract
Nonlinear dynamical systems in applications such as design and control typically depend on a set of variable parameters that represent system geometry, boundary conditions, material properties, etc. Such a parameterized dynamical system requires a parameterized model (e.g., a parameterized differential equation) to describe. On the one hand, to discover the wide variety of the parameter-dependent dynamical behaviors, repeated simulations with the parameterized model are often required over a large range of parameter values, leading to significant computational burdens especially when the system is complex (strongly nonlinear and/or high-dimensional) and the high-fidelity model is inefficient to simulate. Thus, seeking surrogate models that mimic the behaviors of high-fidelity parameterized models while being efficient to simulate is critically needed. On the other hand, the governing equations of the parameterized nonlinear dynamical system (e.g., an aerodynamic system with a physical model (full-scale or scaled in the laboratory) for optimization or design tasks) may be unknown or partially unknown due to insufficient physics knowledge, leading to an inverse problem where we need to identify the models from measurement data only. Accordingly, this work presents a novel deep generative framework for data-driven surrogate modeling/identification of parameterized nonlinear dynamical systems from data only. Specifically, the presented framework learns the direct map** from simulation parameters to visualization images of dynamical systems by leveraging deep generative convolutional neural networks, yielding two advantages: (i) the surrogate simulation is efficient because the calculation of transient dynamics over time is circumvented; (ii) the surrogate output retains characterizing ability and flexibility as the visualization image is customizable and supports any visualization scheme for revealing and representing high-level dynamics feature (e.g., Poincaré map). We study and demonstrate the framework on Lorenz system, forced pendulum system, and forced Duffing system. We present and discuss the prediction performance of the obtained surrogate models. It is observed that the obtained model has promising performance on capturing the sensitive parameter dependence of the nonlinear dynamical behaviors even when the bifurcation occurs. We also discuss in detail the limitation of this work and potential future work.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11071-023-08391-0/MediaObjects/11071_2023_8391_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11071-023-08391-0/MediaObjects/11071_2023_8391_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11071-023-08391-0/MediaObjects/11071_2023_8391_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11071-023-08391-0/MediaObjects/11071_2023_8391_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11071-023-08391-0/MediaObjects/11071_2023_8391_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11071-023-08391-0/MediaObjects/11071_2023_8391_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11071-023-08391-0/MediaObjects/11071_2023_8391_Fig7_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11071-023-08391-0/MediaObjects/11071_2023_8391_Fig8_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11071-023-08391-0/MediaObjects/11071_2023_8391_Fig9_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11071-023-08391-0/MediaObjects/11071_2023_8391_Fig10_HTML.png)
Similar content being viewed by others
Data availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
References
Mack, Y., Goel, T., Shyy, W., Haftka, R.: Surrogate model-based optimization framework: A case study in aerospace design. In: Studies in Computational Intelligence, vol. 51, pp. 323–342. Springer, (2007)
Tornabene, R., Wang, X Y., Christopher J.S., Freeh, J.E.: Development of parametric mass and volume models for an aerospace SOFC/gas turbine hybrid system. In: Proceedings of the ASME Turbo Expo, vol. 5, pp. 5–144. American Society of Mechanical Engineers Digital Collection, (2005)
van Tooren, M.J.L., Krakers, L.A., La Rocca, G., Beukers, A.: Design and technology in aerospace; parametric modelling of complex systems including active components. In: Hahn, HTh., Martin, M.J., (eds.), 14th International conference on composite materials, pp. 1–10. Society of Manufacturing engineers, (2003)
Bhattacharyya, S.P., Keel, L.H.: Robust control: the parametric approach. In: Advances in Control Education 1994, pages 49–52. Elsevier, (1995)
Lee, Ho Jae, Park, ** Bae, Chen, Guanrong: Robust fuzzy control of nonlinear systems with parametric uncertainties. IEEE Trans. Fuzzy Syst. 9(2), 369–379 (2001)
Kulfan, Brenda M.: Universal Parametric Geometry Representation Method. J. Aircr. 45(1), 142–158 (2008)
Brunner, W.: An Iteration Procedure for Parametric Model Building and Boundary Value Problems. In: Papers Presented at the May 9-11, 1961, Western Joint IRE-AIEE-ACM Computer Conference, IRE-AIEE-ACM ’61 (Western), pages 519–533, New York, NY, USA, (1961). Association for Computing Machinery
Meng, **g Hui, Zhang, **n **n, Wang, **ao Dong: Characteristics analysis and parametric study of a thermoelectric generator by considering variable material properties and heat losses. Int. J. Heat Mass Transf. 80, 227–235 (2015)
Song, W., Keane, A.J.: A study of shape parameterisation methods for airfoil optimisation. In: 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference. vol. 4, pp. 2031–2038. Albany, New York (2004)
Keane, A., Forrester, A., Sobester, A.: Engineering design via surrogate modelling: a practical guide. American Institute of Aeronautics and Astronautics, Inc., (2008)
Eldred, M. S., Dunlavy, D. M.: Formulations for surrogate-based optimization with data fit, multifidelity, and reduced-order models. In: Collection of Technical Papers - 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, vol. 4, pp. 2450–2469. American Institute of Aeronautics and Astronautics Inc., (2006)
March, A., Willcox, K.: Provably convergent multifidelity optimization algorithm not requiring high-fidelity derivatives. AIAA J 50(5), 1079–1089 (2012)
Alexandrov, N.M., Lewis, R.M., Gumbert, C.R., Green, L.L., Newman, P.A.: Approximation and model management in aerodynamic optimization with variable-fidelity models. J Aircr 38(6), 1093–1101 (2001)
Kerschen, Gaetan, Golinval, Jean Claude, Vakakis, Alexander F., Bergman, Lawrence A.: The method of proper orthogonal decomposition for dynamical characterization and order reduction of mechanical systems: An overview. Nonlinear Dyn. 41(1–3), 147–169 (2005)
Bui-Thanh, Tan, Willcox, Karen, Ghattas, Omar: Parametric reduced-order models for probabilistic analysis of unsteady aerodynamic applications. AIAA J. 46(10), 2520–2529 (2008)
Hong, Sung Kwon, Epureanu, Bogdan I., Castanier, Matthew P., Gorsich, David J.: Parametric reduced-order models for predicting the vibration response of complex structures with component damage and uncertainties. J. Sound Vib. 330(6), 1091–1110 (2011)
Amsallem, David, Farhat, Charbel: An online method for interpolating linear parametric reduced-order models. SIAM J. Sci. Comput. 33(5), 2169–2198 (2011)
Lee, Chung Hao, Chen, Jiun Shyan: Proper orthogonal decomposition-based model order reduction via radial basis functions for molecular dynamics systems. Int. J. Numer. Methods Eng. 96(10), 599–627 (2013)
Mignolet, Marc P., Przekop, Adam, Rizzi, Stephen A., Michael Spottswood, S.: A review of indirect/non-intrusive reduced order modeling of nonlinear geometric structures. J. Sound Vib. 332(10), 2437–2460 (2013)
Benner, Peter, Gugercin, Serkan, Willcox, Karen: A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Rev. 57(4), 483–531 (2015)
Kaufman, M., Balabanov, V., Burgee, S.L., Giunta, A.A., Grossman, B., Haftka, R.T., Mason, W.H., Watson, L.T.: Variable-complexity response surface approximations for wing structural weight in HSCT design. Comput. Mech. 18(2), 112–126 (1996)
Venter, Gerhard, Haftka, Raphael T., Starnes, James H.: Construction of response surface approximations for design optimization. AIAA J. 36(12), 2242–2249 (1998)
Simpson, Timothy W., Mauery, Timothy M., Korte, John J., Mistree, Farrokh: Kriging models for global approximation in simulation-based multidisciplinary design optimization. AIAA J. 39(12), 2233–2241 (2001)
Kennedy, M.C., O’Hagan, A.: Bayesian calibration of computer models. J. R. Stat. Soc. Ser. B Stat. Method. 63(3), 425–464 (2001)
Eldred, M. S., Giunta, A. A., Collis, S. S.: Second-order corrections for surrogate-based optimization with model hierarchies. In: Collection of Technical Papers - 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, vol. 3, pp. 1754–1768. American Institute of Aeronautics and Astronautics Inc., (2004)
Wild, Stefan M., Regis, Rommel G., Shoemaker, Christine A.: Orbit: optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30(6), 3197–3219 (2008)
Masri, S.F., Caughey, T.K.: A nonparametric identification technique for nonlinear dynamic problems. J. Appl. Mech. 46(2), 433–447 (1979)
Quaranta, Giuseppe, Lacarbonara, Walter, Masri, Sami F.: A review on computational intelligence for identification of nonlinear dynamical systems. Nonlinear Dyn. 99(2), 1709–1761 (2020)
Jianyang, Yu., Zhao Wang, Fu., Chen, Jianing Yu, Wang, Cong: Kriging surrogate model applied in the mechanism study of tip leakage flow control in turbine cascade by multiple DBD plasma actuators. Aerosp. Sci. Technol. 85, 216–228 (2019)
Jung, C.: Poincare map for scattering states. J. Phys. A Math. Gen. 19(8), 1345–1353 (1986)
Bishop, S.R., Xu, D.L., Clifford, M.J.: Flexible control of the parametrically excited pendulum. Proc. R. Soc. A Math. Phys. Eng. Sci. 452(1951), 1789–1806 (1996)
Stachowiak, Tomasz, Okada, Toshio: A numerical analysis of chaos in the double pendulum. Chaos, Solitons and Fractals 29(2), 417–422 (2006)
Dosovitskiy, A., Springenberg, J.T., Brox, T.: Learning to generate chairs with convolutional neural networks. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 07-12-June, pp. 1538–1546, (2015)
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., Lee, H.: Generative adversarial text to image synthesis. In: 33rd International Conference on Machine Learning, ICML 2016, vol. 3, pp. 1681–1690, (2016)
Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., Catanzaro, B.: High-resolution image synthesis and semantic manipulation with conditional GANs. In ar**v, pp. 8798–8807, (2017)
Odena, A., Olah, C., Shlens, J.: Conditional image synthesis with auxiliary classifier gans. In 34th International Conference on Machine Learning, ICML 2017, vol. 6, pp. 4043–4055. PMLR, (2017)
He, Wenbin, Wang, Junpeng, Guo, Hanqi, Wang, Ko Chih, Shen, Han Wei, Raj, Mukund, Nashed, Youssef S.G., Peterka, Tom: InSituNet: deep image synthesis for parameter space exploration of ensemble simulations. IEEE Trans. Vis. Comput. Gr. 26(1), 23–33 (2020)
Kalchbrenner, N., Grefenstette, E., Blunsom, P.: a convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, (2014)
Goodfellow, Ian J., Pouget-Abadie, Jean, Mirza, Mehdi, Bing, Xu., Warde-Farley, David, Ozair, Sherjil, Courville, Aaron, Bengio, Yoshua: Generative adversarial nets. In Adv. Neural Inf. Process. Syst. 3, 2672–2680 (2014)
Mirza, M., Osindero, S.: Conditional generative adversarial nets. ar**v preprint, pages 1–7, (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2016-Decem, pp. 770–778. IEEE Computer Society, (2016)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9906 LNCS, pp. 694–711. Springer Verlag, (2016)
Chen, Haidong, Zhang, Song, Chen, Wei, Mei, Honghui, Zhang, Jiawei, Mercer, Andrew, Liang, Ronghua, Huamin, Qu.: Uncertainty-aware multidimensional ensemble data visualization and exploration. IEEE Trans. Vis. Comput. Gr. 21(9), 1072–1086 (2015)
Bock, A., Pembroke, A., Leila Mays, M., Rastaetter, L., Ynnerman, Anders, R., Timo: Visual verification of space weather ensemble simulations. In: 2015 IEEE Scientific Visualization Conference, SciVis 2015 - Proceedings, pp. 17–24. Institute of Electrical and Electronics Engineers Inc., (2016)
Biswas, Ayan, Lin, Guang, Liu, **aotong, Shen, Han-Wei.: Visualization of time-varying weather ensembles across multiple resolutions. IEEE Trans. Vis. Comput. Gr. 23(1), 841–850 (2017)
Poco, Jorge, Dasgupta, Aritra, Wei, Yaxing, Hargrove, William, Schwalm, Christopher R., Huntzinger, Deborah N., Cook, Robert, Bertini, Enrico, Silva, Claudio T.: Visual reconciliation of alternative similarity spaces in climate modeling. IEEE Trans. Vis. Comput. Gr. 20(12), 1923–1932 (2014)
Bruckner, Stefan, Möller, Torsten: Result-driven exploration of simulation parameter spaces for visual effects design. IEEE Trans. Vis. Comput. Gr. 16(6), 1468–1476 (2010)
Orban, Daniel, Keefe, Daniel F., Biswas, Ayan, Ahrens, James, Rogers, David: Drag and track: a direct manipulation interface for contextualizing data instances within a continuous parameter space. IEEE Trans. Vis. Comput. Gr. 25(1), 256–266 (2019)
Salakhutdinov, R., Hinton, G.: Deep Boltzmann machines. In: David van D. ,Max. W., (eds.), Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics, volume 5 of Proceedings of Machine Learning Research, pp. 448–455, Hilton Clearwater Beach Resort, Clearwater Beach, Florida USA, (2009). PMLR
Kingma, D.P., Mohamed, S., Rezende, D.J., Welling, M.: Semi-supervised learning with deep generative models. In: NIPS, pp. 3581–3589, (2014)
Smolensky, P.: Information Processing in Dynamical Systems: Foundations of Harmony Theory. Technical report, Colorado Univ at Boulder Dept of Computer Science, (1986)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Lee, H., Grosse, R., Ranganath, R., Ng, Andrew Y.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the 26th Annual International Conference on Machine Learning - ICML ’09, pages 1–8, New York, New York, USA, (2009). Association for Computing Machinery (ACM)
Ranzato, M., Susskind, J., Mnih, V., Hinton, G.: On deep generative models with applications to recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 2857–2864. IEEE Computer Society, (2011)
Memisevic, R., Hinton, G.: Unsupervised learning of image transformations. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognitio, (2007)
Theis, Lucas, Hosseini, Reshad, Bethge, Matthias: Mixtures of conditional gaussian scale mixtures applied to multiscale image representations. PLoS ONE 7(7), e39857 (2012)
Larochelle, H., Murray, I.: The Neural Autoregressive Distribution Estimator. In: Geoffrey, G., David, D., Miroslav D., (eds.), Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pages 29–37, Fort Lauderdale, FL, USA, (2011). PMLR
Tang, Yichuan, Salakhutdinov, Ruslan: Learning Stochastic Feedforward Neural Networks. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Vol.1, NIPS’13, pp. 530–538, Red Hook, NY, USA, (2013). Curran Associates Inc
Kingma, D.P., Welling., M.: Auto-encoding variational bayes (2014)
Rezende, D. J., Mohamed, S., Wierstra, D.: Stochastic Backpropagation and Approximate Inference in Deep Generative Models. In: Eric, P. X., Tony, J. (eds.), Proceedings of the 31st International Conference on Machine Learning, vol. 32 of Proceedings of Machine Learning Research, pp. 1278–1286, Be**g, China, (2014). PMLR
Bengio, Y., Laufer, E., Alain, G., Yosinski, J.: Deep generative stochastic networks trainable by backprop. In Eric, P. X., Tony, J. (eds.), Proceedings of the 31st International Conference on Machine Learning, vol. 32 of Proceedings of Machine Learning Research, pp. 226–234, Be**g, China, (2014). PMLR
Huang, K., Wang, Y., Tao, M., Zhao, T.: Why do deep residual networks generalize better than deep feedforward networks? — a neural tangent kernel perspective. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.), Advances in Neural Information Processing Systems, volume 33, pages 2698–2709. Curran Associates, Inc., (2020)
He, Fengxiang, Liu, Tongliang, Tao, Dacheng: Why resnet works? residuals generalize. IEEE Trans. Neural Netw. Learn. Syst. 31(12), 5349–5362 (2020)
Jacot, A., Gabriel, F., Hongler, C.: Neural Tangent Kernel: Convergence and Generalization in Neural Networks. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.), Advances in neural information processing systems, vol. 31. Curran Associates, Inc., (2018)
Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: 32nd International Conference on Machine Learning, ICML 2015, vol. 1, pp.448–456. International Machine Learning Society (IMLS), ( 2015)
Miyato, T., Koyama, M.: cGANs with projection discriminator. In: International Conference on Learning Representations, (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, (2015)
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz,J., Wang, Z., Shi, W.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10, pp. 807-814, Madison, WI, USA, (2010). Omnipress
Hou, X., Shen, L., Sun, K., Qiu, G.: Deep feature consistent variational autoencoder. In: Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017, pp. 1133–1141. Institute of Electrical and Electronics Engineers Inc., (2017)
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I.J., Harp, A., Irving, G., Isard, M., Jia, Y., Józefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané,D., Monga, R., Moore, S., Murray, D.G., Olah,C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P.A., Vanhoucke, V., Vasudevan, V., Viégas, F.B., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: Tensorflow: large-scale machine learning on heterogeneous distributed systems. CoRR, arxiv:1603.04467, (2016)
Kingma, D.P., Ba, Jimmy Lei, A.: A method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. International Conference on Learning Representations, ICLR, (2015)
Raissi, Maziar, Perdikaris, Paris, Karniadakis, George E.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)
Li, Shanwu, Yang, Yongchao: Data-driven identification of nonlinear normal modes via physics-integrated deep learning. Nonlinear Dyn. 106(4), 3231–3246 (2021)
Pei, **-Song., Wright, Joseph P., Smyth, Andrew W.: Map** polynomial fitting into feedforward neural networks for modeling nonlinear dynamic systems and beyond. Comput. Methods Appl. Mech. Eng. 194(42), 4481–4505 (2005)
Pei, **-Song., Mai, Eric C., Wright, Joseph P., Masri, Sami F.: Map** some basic functions and operations to multilayer feedforward neural networks for modeling nonlinear dynamical systems and beyond. Nonlinear Dyn. 71(1), 371–399 (2013)
Funding
This research was partially funded by the Physics of Artificial Intelligence Program of U.S. Defense Advanced Research Projects Agency (DARPA) and the Michigan Technological University faculty startup fund.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Code availability
The codes generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, S., Yang, Y. A deep generative framework for data-driven surrogate modeling and visualization of parameterized nonlinear dynamical systems. Nonlinear Dyn 111, 10287–10307 (2023). https://doi.org/10.1007/s11071-023-08391-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11071-023-08391-0