Abstract
We propose a novel Bayesian Ridge Echo State Network (BRESN) model for nonlinear time series prediction, based on Bayesian Ridge Regression and Independent Component Analysis. BRESN has a regularization effect to avoid over-fitting, at the same time being robust to noise owing to its probabilistic strategy. In BRESN we also use Independent Component Analysis (ICA) for dimensionality reduction, and show that ICA improves the model’s accuracy more than other reduction techniques. Furthermore, we evaluate the proposed model on both synthetic and real-world datasets to compare its accuracy with twelve combinations of four other regression models and three different choices of dimensionality reduction techniques, and measure its running time. Experimental results show that our model significantly outperforms other state-of-the-art ESN prediction models while maintaining a satisfactory running time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bishop, C.M.: Training with noise is equivalent to Tikhonov regularization. Neural Comput. 7(1), 108–116 (1995)
Bishop, C.M., Tip**, M.E.: Bayesian regression and classification. Nato Science Series sub Series III Computer And Systems Sciences, vol. 190, pp. 267–288 (2003)
Han, M., Mu, D.: Multi-reservoir echo state network with sparse Bayesian learning. In: Zhang, L., Lu, B.-L., Kwok, J. (eds.) ISNN 2010. LNCS, vol. 6063, pp. 450–456. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13278-0_58
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1–3), 489–501 (2006)
Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. German National Research Center for Information Technology GMD, Bonn, Germany, Technical report, vol. 148(34), p. 13 (2001)
Jaeger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)
Kantz, H., Schreiber, T.: Nonlinear Time Series Analysis, vol. 7. Cambridge University Press, Cambridge (2004)
Koskela, T., Lehtokangas, M., Saarinen, J., Kaski, K.: Time series prediction with multilayer perceptron, FIR and Elman neural networks. In: Proceedings of the World Congress on Neural Networks, pp. 491–496. Citeseer (1996)
Leung, H., Lo, T., Wang, S.: Prediction of noisy chaotic time series using an optimal radial basis function neural network. IEEE Trans. Neural Netw. 12(5), 1163–1172 (2001)
Løkse, S., Bianchi, F.M., Jenssen, R.: Training echo state networks with regularization through dimensionality reduction. Cogn. Comput. 9(3), 364–378 (2017)
Reinsel, G.C.: Elements of Multivariate Time Series Analysis. Springer, New York (2003)
Shi, Z., Han, M.: Support vector echo-state machine for chaotic time-series prediction. IEEE Trans. Neural Netw. 18(2), 359–372 (2007)
SILSO World Data Center: The international sunspot number. International Sunspot Number Monthly Bulletin and online catalogue (1749-2017). http://www.sidc.be/silso/
Takens, F.: Detecting strange attractors in turbulence. In: Rand, D., Young, L.-S. (eds.) Dynamical Systems and Turbulence, Warwick 1980. LNM, vol. 898, pp. 366–381. Springer, Heidelberg (1981). https://doi.org/10.1007/BFb0091924
Acknowledgments
This research was supported by the MSIT (Ministry of Science, ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2018-2018-1-00877) supervised by the IITP (Institute for Information & communications Technology Promotion), and International Research & Development Program of the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning of Korea (2016K1A3A7A03952054).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Nguyen, H.M., Kalra, G., Jun, T.J., Kim, D. (2018). A Novel Echo State Network Model Using Bayesian Ridge Regression and Independent Component Analysis. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds) Artificial Neural Networks and Machine Learning – ICANN 2018. ICANN 2018. Lecture Notes in Computer Science(), vol 11140. Springer, Cham. https://doi.org/10.1007/978-3-030-01421-6_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-01421-6_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01420-9
Online ISBN: 978-3-030-01421-6
eBook Packages: Computer ScienceComputer Science (R0)