SpringerLink
Log in

Multi-scale deep echo state network for time series prediction

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Echo state network (ESN) has widely attracted many researchers due to its training process without backpropagation. However, it is hard for single ESN to fit those complex and polytrophic situations. Under this case, a novel multiscale deep ESN (MDESN) is developed in this study, which integrates the deep learning, parallel structure and multiscale reservoir state matrix mode, respectively. The deep framework is responsible for collecting multivariate dimensional reservoir states; Upgradation of various reservoir states could efficiently dig hidden characteristics for decoding. The parallel structure could be trained simultaneously to decrease time consumption through the multiple thread method. According to the multivariate high dimensional map** method, MDESN could obtain more robust generalization information compared to the state-of-the-art models. MDESN is evaluated in two chaos prediction benchmarks (Lorenz and Mackey-Glass) and real solar irradiance predictions. Various prediction horizons including one-step-ahead and multi-step-ahead prediction are designed and conducted respectively to verify the effectiveness and robustness of MDESN. Furthermore, RMSE, MAE, MAPE, and R are adopted to evaluate our proposed model. The statistical results demonstrate that MDESN has the best-performing adaptability and robustness compared to classical ESN, deep ESN (DESN), and other state-of-the-art models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Algorithm 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability statement

Data are available on request from the authors.

References

  1. Wang H, Liu Y, Lu P et al (2022) Echo state network with logistic map** and bias dropout for time series prediction. Neurocomputing 489:196–210

    Article  Google Scholar 

  2. Hua Y, Zhao Z, Li R et al (2019) Deep learning with long short-term memory for time series prediction. IEEE Commun Mag 57(6):114–119

    Article  Google Scholar 

  3. Livieris IE, Pintelas E, Pintelas P (2020) A CNN-LSTM model for gold price time-series forecasting. Neural Comput Appl 32:17351–17360

    Article  Google Scholar 

  4. Jiang R, Zeng S, Song Q et al (2022) Deep-chain echo state network with explainable temporal dependence for complex building energy prediction. IEEE Trans Ind Inform 19(1):426–435

    Article  Google Scholar 

  5. Guo X, Zhang X, Lu W et al (2022) Real-time prediction of 6-DOF motions of a turret-moored FPSO in harsh sea state. Ocean Eng 265:112500

    Article  Google Scholar 

  6. Cuong Q, Hung N, Thomas V (2022) A Kalman filter based ARX time series modeling for force identification on flexible manipulators. Mech Syst Signal Process 169:108743

    Article  Google Scholar 

  7. Lee YS, Tong LI (2011) Forecasting time series using a methodology based on autoregressive integrated moving average and genetic programming. Knowl-Based Syst 24(1):66–72

    Article  Google Scholar 

  8. Yang H, Pan Z, Tao Q et al (2018) Online learning for vector autoregressive moving-average time series prediction. Neurocomputing 315:9–17

    Article  Google Scholar 

  9. Sapankevych NI, Sankar R (2009) Time series prediction using support vector machines: a survey. IEEE Comput Intell Mag 4(2):24–38

    Article  Google Scholar 

  10. Yang D, Ye Z, Lim L et al (2015) Very short term irradiance forecasting using the lasso. Sol Energy 114:314–326

    Article  Google Scholar 

  11. Lai R, Fan C, Huang W et al (2009) Evolving and clustering fuzzy decision tree for financial time series data forecasting. Expert Syst Appl 36(2):3761–3773

    Article  Google Scholar 

  12. Cai Y, Lin H, Zhang M (2019) Map** paddy rice by the object-based random forest method using time series Sentinel-1/Sentinel-2 data. Adv Space Res 64(11):2233–2244

    Article  Google Scholar 

  13. Li L, Dai S, Cao Z et al (2020) Using improved gradient-boosted decision tree algorithm based on Kalman filter (GBDT-KF) in time series prediction. J Supercomput 76(9):6887–6900

    Article  Google Scholar 

  14. Vishwakarma GK, Paul C, Elsawah AM (2022) A hybrid feedforward neural network algorithm for detecting outliers in non-stationary multivariate time series. Expert Syst Appl 184:115545

    Article  Google Scholar 

  15. Wang W, Liu W, Chen H (2020) Information granules-based BP neural network for long-term prediction of time series. IEEE Trans Fuzzy Syst 29(10):2975–2987

    Article  Google Scholar 

  16. Durairaj DM, Mohan BHK (2022) A convolutional neural network based approach to financial time series prediction. Neural Comput Appl 34(16):13319–13337

    Article  Google Scholar 

  17. Niu H, Wang J (2014) Financial time series prediction by a random data-time effective RBF neural network. Soft Comput 18(3):497–508

    Article  Google Scholar 

  18. Chu X, ** H, Li Y et al (2021) CDA-LSTM: an evolutionary convolution-based dual-attention LSTM for univariate time series prediction. Neural Comput Appl 33:16113–16137

    Article  Google Scholar 

  19. Wu Z, Li Q, **a X (2020) Multi-timescale forecast of solar irradiance based on multi-task learning and echo state network approaches. IEEE Trans Ind Inform 17(1):300–310

    Article  Google Scholar 

  20. Yao X, Shao Y, Fan S et al (2022) Echo state network with multiple delayed outputs for multiple delayed time series prediction. J Frankl Inst 359(18):11089–11107

    Article  MathSciNet  Google Scholar 

  21. Jaeger H, Haas H (2004) Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304:78–80

    Article  Google Scholar 

  22. Wu Z, Zeng S, Jiang R et al (2023) Explainable temporal dependence in multi-step wind power forecast via decomposition based chain echo state networks. Energy 270:126906

    Article  Google Scholar 

  23. Liu H, Zhang X (2021) AQI time series prediction based on a hybrid data decomposition and echo state networks. Environ Sci Pollut Res 28:51160–51182

    Article  Google Scholar 

  24. Li D, Han M, Wang J (2012) Chaotic time series prediction based on a novel robust echo state network. IEEE Trans Neural Netw Learn Syst 23(5):787–799

    Article  Google Scholar 

  25. Kim T, King BR (2020) Time series prediction using deep echo state networks. Neural Comput Appl 32:17769–17787

    Article  Google Scholar 

  26. Chen Q, Shi L, Na J et al (2018) Adaptive echo state network control for a class of pure-feedback systems with input and output constraints. Neurocomputing 275:1370–1382

    Article  Google Scholar 

  27. Ma Q, Shen L, Chen W et al (2016) Functional echo state network for time series classification. Inf Sci 373:1–20

    Article  Google Scholar 

  28. Gallicchio C, Micheli A, Pedrelli L (2018) Design of deep echo state networks. Neural Netw 108:33–47

    Article  Google Scholar 

  29. Li Q, Wu Z, Ling R et al (2020) Multi-reservoir echo state computing for solar irradiance prediction: a fast yet efficient deep learning approach. Appl Soft Comput J 95:106481

    Article  Google Scholar 

  30. Ma Q, Shen L, Cottrell GW (2020) DeePr-ESN: a deep projection-encoding echo-state network. Inf Sci 511:152–171

    Article  Google Scholar 

  31. Gao SH, Cheng MM, Zhao K et al (2019) Res2net: a new multi-scale backbone architecture. IEEE Trans Pattern Anal Mach Intell 43(2):652–662

    Article  Google Scholar 

  32. Song T, Zhang X, Ding M et al (2022) DeepFusion: a deep learning based multi-scale feature fusion method for predicting drug-target interactions. Methods 204:269–277

    Article  Google Scholar 

  33. Ma W, Wu Y, Cen F et al (2020) Mdfn: multi-scale deep feature learning network for object detection. Pattern Recogn 100:107149

    Article  Google Scholar 

  34. Guo C, Fan B, Zhang Q et al (2020) Augfpn: improving multi-scale feature learning for object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12595–12604

  35. Zhang R, **e P, Wang C et al (2019) Classifying transportation mode and speed from trajectory data via deep multi-scale learning. Comput Netw 162:106861

    Article  Google Scholar 

  36. Yao Y, Zhang S, Yang S et al (2020) Learning attention representation with a multi-scale CNN for gear fault diagnosis under different working conditions. Sensors 20(4):1233

    Article  Google Scholar 

  37. Wang X, Wang M (2008) A hyperchaos generated from Lorenz system. Physica A 387(14):3751–3758

    Article  MathSciNet  Google Scholar 

  38. Samanta B (2011) Prediction of chaotic time series using computational intelligence. Expert Syst Appl 8(9):11406–11411

    Article  Google Scholar 

  39. http://cimis.water.ca.gov

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (62163026, 62303206, 52178271), the Natural Science Foundation of Jiangxi Province (20224BAB212018, 20224BAB212019), and the National Key Research and Development Program of China (2022YFE0198900, 2021YFF0500903).

Author information

Authors and Affiliations

  1. College of Advanced Manufacturing, Nanchang University, Nanchang, 330031, China

    Tao Li & Qian Li

  2. College of Information Engineering, Nanchang University, Nanchang, 330031, China

    Zhijun Guo

  3. College of Automation, Chongqing University, Chongqing, 400044, China

    Zhou Wu

Authors
  1. Tao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhijun Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhou Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Qian Li or Zhou Wu.

Ethics declarations

Conflict of interest

The authors declare that they have no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, T., Guo, Z., Li, Q. et al. Multi-scale deep echo state network for time series prediction. Neural Comput & Applic 36, 13305–13325 (2024). https://doi.org/10.1007/s00521-024-09761-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09761-4

Keywords

Search

Navigation