Abstract
This work analyzes the use of artificial neural networks in the short-term streamflow forecasting for large interconnected hydropower systems. The state-of-the-art optimization algorithms, activation functions, and weight initialization techniques are investigated together with classic methods. We present an algorithm to define the neural network inputs in large hydrosystems and apply it to create models for 55 major hydro plants located in the Paraná Basin, which contribute to more than 30% of the total power generated in Brazil. The paper also compares the performance of the neural networks with the hydrological models that are currently used by the independent system operator to define the dispatch of the electric power generators. Our results show that, overall, the neural network models provide more accurate forecasts than the hydrological models used by the Brazilian System Operator. Finally, the paper discusses the contributions of historical rainfall information in the forecasting of streamflow while using neural network models.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and material
The data used in this research are publicly available on the internet.
Code availability
Available upon request.
Abbreviations
- Adam:
-
Adaptive moment estimation
- ANN:
-
Artificial neural network
- AR:
-
Autoregressive
- ARMA:
-
Autoregressive moving average
- CPU:
-
Central processing unit
- GD:
-
Gradient descent
- GDM:
-
Gradient descent with momentum
- GPU:
-
Graphics processing unit
- ISO:
-
Independent system operator
- IVS:
-
Input variable selection
- MAPE:
-
Mean absolute percentage error
- MGB-IPH:
-
Large-scale hydrological model
- MLP:
-
Multi-layer perceptron
- MSE:
-
Mean square error
- NSE:
-
Nash–Sutcliffe efficiency
- OWI:
-
Optimal weight initialization
- PAR:
-
Periodic autoregressive
- PARMA:
-
Periodic autoregressive moving average
- ReLU:
-
Rectified linear unit
- RMSprop:
-
Root-mean-square propagation
- SMAP:
-
Soil moisture accounting procedure
- SVM:
-
Support vector machine
- VELMA:
-
Visualizing ecosystem land management assessments
- WI:
-
Weight initialization
- \({\text{In}}\) :
-
Set of input candidates
- \({\text{In}}_{{\text{S}}}^{*}\) :
-
Best set of streamflow inputs used in a specific ANN
- \({\text{Is}}_{{t_{{\text{I}}} }}\) :
-
Input streamflow candidate (\(s_{{\text{I}}}\), \(t_{{\text{I}}}\))
- \({\text{Ip}}_{{t_{{\text{I}}} }}^{{D_{{{\text{AR}}}} }}\) :
-
Input rainfall candidate accumulated through \(D_{{{\text{AR}}}}\) days (\(p_{{\text{I}}}\), \(t_{{\text{I}}}\)); \(D_{{{\text{AR}}}} = 1\): nonaccumulated rainfall
- \({\text{Os}}_{{t_{{\text{o}}} }}\) :
-
Output streamflow (\(s_{{\text{O}}}\), \(t_{{\text{O}}}\))
- \(p_{{\text{I}}} \in {\text{In}}\_ p\) :
-
Set of rainfall stations considered for input candidates
- \(s_{{\text{I}}} \in {\text{In}}\_s\) :
-
Set of streamflow stations considered for input candidates
- \(s_{{\text{O}}}\) :
-
Output streamflow station
- \(t_{{\text{I}}}\) :
-
Number of days preceding the first day of forecast
- \(t_{{\text{O}}}\) :
-
Day of forecast
- \(C\) :
-
Component of the cost function that penalizes the difference between \(y^{\left( i \right)}\) and \(\hat{y}^{\left( i \right)}\)
- \({\text{Cr}}\left( {A,B} \right)\) :
-
Function that computes the correlation coefficient between the variables \(A\) and \(B\)
- \(E_{{{\text{dev}}}} \left( k \right)\) :
-
Error in the development set until the \(k{\text{th}}\) epoch simulation
- \(J\) :
-
Cost function with \(\ell_{2}\) regularization
- \(M2\) :
-
Cost function (28)
- \(R2\) :
-
Cost function (29)
- \(U\left( {a_{1} ,a_{2} } \right)\) :
-
Continuous uniform distribution in the interval [\(a_{1}\), \(a_{2}\)]
- \(U_{{\text{D}}} \left( {a_{1} ,a_{2} } \right)\) :
-
Discrete uniform distribution in the interval [\(a_{1}\), \(a_{2}\)]
- \({\text{Var}}\left[ {w^{\ell } } \right]\) :
-
Variance of the weights in the \(\ell {\text{th}}\) layer
- \(w_{2}^{2}\) :
-
Square of the \(\ell_{2}\) norm of the weight matrixes
- \(\sigma \left( x \right)\) :
-
Logistic sigmoid activation function
- \(b\) :
-
Bias vector
- \(C_{{\text{IS/OS}}}\) :
-
Lowest correlation coefficient between the streamflow input candidates and output streamflow
- \(C_{{{\text{IR}}}}\) :
-
Maximum correlation coefficient for the input rainfall candidates
- \(C_{{\text{IR/OS}}}\) :
-
Lowest correlation coefficient between the rainfall input candidates and output streamflow
- \(C_{{{\text{IS}}}}\) :
-
Maximum correlation coefficient for the input streamflow candidates
- \(db\) :
-
Gradients of the bias vector
- \(dw\) :
-
Gradients of the weights
- \(D_{{{\text{AR}}}}\) :
-
Number of days lag that rainfall is accumulated
- \(D_{{{\text{IS}}}}\) :
-
Number of days lag of input streamflow investigated
- \(D_{{{\text{OS}}}}\) :
-
Number of days of output streamflow
- \(D_{{{\text{IR}}}}\) :
-
Number of days lag of input rainfall investigated
- \(\overline{{{\text{Epoch}}}}\) :
-
Maximum number of epochs
- \(k_{{{\text{prob}}}}\) :
-
Regularization hyperparameter in dropout
- \(m\) :
-
Number of output variables
- \(n\) :
-
Number of output examples
- \(n_{\ell }\) :
-
Number of neurons in the \(\ell {\text{th}}\) layer
- \(N_{{{\text{Hyp}}}}\) :
-
Number of times that different hyperparameters are tested
- \(N_{{{\text{RC}}}}\) :
-
Number of times that different \(C_{{\text{IR/OS}}}\) and \(C_{{{\text{IR}}}}\) correlation coefficients are tested
- \(N_{{{\text{SC}}}}\) :
-
Number of times that different \(C_{{\text{IS/OS}}}\) and \(C_{{{\text{IS}}}}\) correlation coefficients are tested
- \(w\) :
-
Weights
- \(\overline{{y^{\left( i \right)} }}\) :
-
Average streamflow for the \(i{\text{th}}\) example
- \(y^{{\left( {i, t} \right)}}\) :
-
Estimate for the \(t{\text{th}}\) output variable (day of forecast) in the \(i{\text{th}}\) example
- \(\hat{y}^{{\left( {i,t} \right)}}\) :
-
Expected value for the \(t{\text{th}}\) output variable (day of forecast) in the \(i{\text{th}}\) example
- \(\alpha\) :
-
Learning rate variable
- \(\lambda\) :
-
\(\ell_{2}\) Regularization hyperparameter
- \(\mu_{0}\) :
-
Mean of the output variables \(\hat{y}\)
- \(\sigma_{0}\) :
-
Standard deviation of the output variables \(\hat{y}\)
References
ABRACEEL—Brazilian Association of The Energy Traders (2018) Annual Report on the Abraceel Activities (in portuguese)
Aggarwal CC (2018) Neural networks and deep learning: a textbook. Springer, Berlin
Akhtar MK, Corzo GA, Andel SJ, Jonoski A (2009) River flow forecasting with artificial neural networks using satellite observed precipitation pre-processed with flow length and travel time information: case study of the Ganges river basin. Hydrol Earth Syst Sci 13:1607–1618
ANA—Brazilian National Water Agency (2019) Operational data from the reservoirs of the Brazilian electric system, viewed 10 January 2021. www.ana.gov.br (in portuguese)
Aytek A, Asce M, Alp M (2008) An application of artificial intelligence for rainfall-runoff modeling. J Earth Syst Sci 117:145–155
Bengio Y (2012) Practical recommendations for gradient-based training of deep architectures. Neural networks: tricks of the trade. Springer, Berlin, pp 437–478
Box GEP, Jenkins GM, Reinsel GC, Ljung GM (2015) Time series analysis forecasting and control, 5th edn. Wiley, London
Bravo JM, Paz AR, Collischonn W, Uvo CB, Pedrollo OC, Chou SC (2009) Incorporating forecasts of rainfall in two hydrologic models used for medium-range streamflow forecasting. J Hydrol Eng 14(5):435–445
Chen L, Singh VP, Guo S, Zhou J, Ye L (2014) Copula entropy coupled with artificial neural network for rainfall–runoff simulation. Stoch Environ Res Risk Assess 28:1755–1767
Clevert D, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (ELUs). CoRR abs/1511.0
CPTEC—Center for weather forecasting and climate studies of the Brazilian National Institute for Space Research (2019) MERGE, viewed 10 January 2021. http://ftp.cptec.inpe.br/modelos/io/produtos/MERGE/
de Queiroz AR, Oliveira FA, Lima JWM, Balestrassi PP (2007) Simulating electricity spot prices in brazil using neural network and design of experiments. IEEE Lausanne Power Tech
de Queiroz AR, Lima LMM, Lima JWM, da Silva BC, Scianni LA (2016) Climate change impacts in the energy supply of the Brazilian hydro-dominant power system. Renew Energy 99:379–389
Dozat T (2016) Incorporating nesterov momentum into adam. In: International conference on learning representations (ICLR)
ECMWF—European Center for Medium-Range Weather Forecasts (2019) viewed 10 January 2021. www.ecmwf.int
Egawa T, Suzuki K, Ichikawa Y, Lizaka T, Matsui T, Shikagawa Y (2011) A water flow forecasting for dam using neural networks and regression models. IEEE Power and Energy Society General Meeting.
Eslamian S (2014) Handbook of Engineering hydrology: modeling, climate change, and variability. CRC Press, London
Faria VAD, de Queiroz AR, Lima LMM, Lima JWM (2018) Cooperative game theory and last addition method in the allocation of firm energy rights. Appl Energy 226:905–915
Galelli S, Humphrey GB, Maier HR, Castelletti A, Dandy GC, Gibbs MS (2014) An evaluation framework for input variable selection algorithms for environmental data-driven models. Environ Model Softw 62:33–51
Géron A (2017) Hands-on machine learning with Scikit-Learn and TensorFlow. O’ Reily Media
Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. Proc AISTATS 9:249–256
Glorot X, Bordes A, Benigo Y (2011) Deep sparse rectifier neural networks. AISTATS 15:315–323
Glorot X, Bordes A, Benigo Y (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958
Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, New York
He K, Zhang X, Ren S, Sun J (2015) Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. ICCV
Hinton GE, Deng L, Yu D, Dahl G, Mohamed A, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath T, Kingsbury B (2012a) Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process Mag 29:82–97
Hinton GE, Srivastava N, and Swersky K (2012b) Lecture 6a overview of mini–batch gradient descent. Coursera Lecture slides, viewed 10 January 2021. https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf
Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012c) Improving neural networks by preventing co-adaptation of feature detectors. ar**v:1207.0580v1
Hsu K, Gupta HV, Sorooshian S (1995) Artificial neural network modeling of the rainfall-runoff process. Water Resour Res 31:2517–2530
Iha—International Hydropower Association (2018) Hydropower status report
Jo T, Hou J, Eickholt J, Cheng J (2015) Improving protein fold recognition by deep learning networks. Sci Rep 5:1–11
Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: International conference on learning representations (ICLR)
KİŞİ Ö (2007) Streamflow forecasting using different artificial neural network algorithms. J Hydrol Eng 12(5):532–539
Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In: NIPS
Kumar DN, Raju KS, Sathish T (2004) River flow forecasting using recurrent neural networks. Water Resour Manag 18:143–161
LeCun Y, Bottou L, Orr GB, Muller KR (1998) Efficient backprop. In: Neural networks, tricks of the trade, Lecture Notes in Computer Science LNCS 1524
Legates DR, McCabe JG (1999) Evaluating the use of goodness-of-fit measurements in hydrologic and hydroclimatic model validation. Water Resour Res 35:233–241
Li X, Shang W, Wang S (2018) Text-based crude oil price forecasting: a deep learning approach. Int J Forecast 35(5):1548–1560
Lima MMM, Popova E, Damien P (2014) Modeling and forecasting of Brazilian reservoir inflows via dynamic linear models. Int J Forecast 30:464–476
Lopes I, Montenegro AAA (2017) Hydrological process simulation at plot scale using the smap model in the semiarid 3:78–86
Lopes JEG, Braga BPF, Conejo JGL (1982) SMAP—a simplified hydrological model. In: Applied modeling in catchment hydrology Water Resources Publications, Littleton, pp 167–176
Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: International conference on machine learning (ICML)
May R, Dandy G, Maier H (2011) Review of input variable selection methods for artificial neural networks artificial neural networks—methodological advances and biomedical applications. InTech
Mishra SK, Singh VP (2003) Soil conservation service curve number (SCS-CN) methodology. Springer, Netherlands
Moriasi DN, Arnold JG, Van MWL, Bingner RL, Harmel RD, Veith TL (2007) Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Trans Am Soc Agric Biol Eng 50:885–900
Morse G, Stanley KO (2016) Simple evolution optimization can rival stochastic gradient descent in neural networks. In: Proceedings of the genetic and evolutionary computation conference.
Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. J Big Data 2:1–21
NOAA—National Centers for Environmental Information (2019), viewed 10 January 2021. <www.ncdc.noaa.gov>
ONS—Brazilian Independent System Operator (2014) Annual Report of Streamflow Forecasting 2014, viewed 10 January 2021. www.ons.org.br (in portuguese)
ONS—Brazilian Independent System Operator (2015) Annual report of streamflow forecasting 2015, viewed 10 January 2021. www.ons.org.br (in portuguese)
ONS—Brazilian Independent System Operator (2016) Annual report of streamflow forecasting 2016, viewed 10 January 2021. www.ons.org.br (in portuguese)
ONS—Brazilian Independent System Operator (2017) Annual report of streamflow forecasting 2017, viewed 10 January 2021. www.ons.org.br (in portuguese)
Polyak BT (1964) Some methods of speeding up the convergence of iteration methods. USSR Comput Math Math Phys 4:1–17
Pontes PRM, Fan FM, Fleischmann AS, Rodrigo CDP, Buarque DC, Siqueira VA, Jardim PF, Sorribas MV, Collischonn W (2017) MGB-IPH model for hydrological and hydraulic simulation of large floodplain river systems coupled with open source GIS. Environ Model Softw 94:1–20
Quan H, Srinivasan D, Khosravi A (2014) Short-term load and wind power forecasting using neural network-based prediction intervals. IEEE Trans Neural Netw Learn Syst 25:303–315
Rasmussen PF, Salas JD, Fagherazzi L, Rassam J, Bobée B (1996) Estimation and validation of contemporaneous PARMA model for streamflow simulation. Water Resour Res 32:3151–3160
REN21 (2018) Renewables 2018 Global Status Report
Rozante JR, Moreira DS, Goncalves LGG, Vila DA (2010) Combining TRMM and surface observations of precipitation: technique and validation over South America (2010) Weather and Forecasting 25:885–894
Ruder S (2017) An overview of gradient descent optimization algorithms. CoRR. ar**v:1609.04747v2.
Russom P (2011) Big data analytics. TDWI best practices report
Santos CAG, Silva GBL (2014) Daily streamflow forecasting using a wavelet transform and artificial neural network hybrid models. Hydrol Sci J 59(2):312–324
Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117
Sitterson J, Knightes C, Parmar R, Wolfe K, Muche M, Avant B (2017) An overview of rainfall-runoff model types. EPA—United States Environmental Protection Agency
Souza FC, Legey LFL (2008) Brazilian electricity market structure and risk management tools. IEEE PES General Meeting.
Souza SA, Costa FS, Xavier LNR, Maceira MEP, and Damázio JM (2010) PREVIVAZ—improving weekly streamflow time series forecasts with current hydrologic state of the river basin. International Workshop Advances in Statistical Hydrology
Sttari MT, Yurekli K, Pal M (2012) Performance evaluation of artificial neural network approaches in forecasting reservoir inflow. Appl Math Model 36:2649–2657
Taormina R, Chau K (2015) Data-driven input variable selection for rainfall-runoff modeling using binary-coded particle swarm optimization and extreme learning. J Hydrol 529:1617–1631
TensorFlow (2019) An open-source machine learning framework for everyone. www.tensorflow.org. Accessed 10 Jan 2021 (in portuguese)
Tian Y, Pei K, Jama S, and Ray B (2018) DeepTest: automated testing of deep-neural-network-driven autonomous cars. In: ICSE '18 proceedings of the 40th international conference on software engineering, pp 303–314
Tongal H, Booij MJ (2018) Simulation and forecasting of streamflows using machine learning models couple with base flow separation. J Hydrol 564:266–282
Wan L, Zeiler M, Zhang S, LeCun Y, Fergus R (2013) Regularization of neural networks using drop connect. In: International conference on machine learning (ICML)
Witten IH, Frank E, Hall MA, Pal CJ (2017) Data mining practical machine learning tools and techniques, 4th edn. Morgan Kaufmann, Burlington
Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. CoRR, abs/1505.00853
Yaseen ZM, El-shafie A, Jaafar O, Afan HA, Sayl KN (2015) Artificial intelligence based model for stream-flow forecasting: 2000–2015. J Hydrol 530:829–844
Yaseen ZM, Jaafar O, Deo RC, Kisi O, Adamowski J, Quilty J, El-Shafie A (2016) Stream-flow forecasting using extreme learning machines: a case study in a semi-arid region in Iraq. J Hydrol 542:603–614
Zadeh MR, Amin S, Khalili D, Singh VP (2010) Daily outflow prediction by multi layer perceptron with logistic sigmoid and tangent sigmoid activation functions. Water Resour Manag 24:2673–2688
Zealand CM, Burn DH, Simonovic SP (1999) Short term streamflow forecasting using artificial neural networks. J Hydrol 214:32–48
Acknowledgements
The authors would like to express their gratitude to AES Tietê for the financial support of the ANEEL Strategic R&D Project 0064-1052/2017. The authors also would like to thank Eduardo H.S. Silva, Lívia M.P. Gazzi, Eliude P. Ferro, and Lanai Torres for technical discussions during the course of this project.
Funding
This research was funded by AES Tietê Brazil under the ANEEL Strategic R&D Project 0064-1052/2017.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by VF and AdQ. The first draft of the manuscript was written by VF, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Additional information
Editorial responsibility: Maryam Shabani.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
de Faria, V.A.D., de Queiroz, A.R., Lima, L.M. et al. An assessment of multi-layer perceptron networks for streamflow forecasting in large-scale interconnected hydrosystems. Int. J. Environ. Sci. Technol. 19, 5819–5838 (2022). https://doi.org/10.1007/s13762-021-03565-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13762-021-03565-y