Abstract
In this work we present a new hybrid algorithm for feedforward neural networks, which combines unsupervised and supervised learning. In this approach, we use a Kohonen algorithm with a fuzzy neighborhood for training the weights of the hidden layers and gradient descent method for training the weights of the output layer. The goal of this method is to assist the existing variable learning rate algorithms. Simulation results show the effectiveness of the proposed algorithm compared with other well-known learning methods.
Similar content being viewed by others
References
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. Parallel distributed processing: explorations in the microstructures of cognition. MIT Press, 1:318–362
Jacobs RA (1988) Increased rates of convergence through learning rate adaptation. Neural Netw 1:295–307
Silva F, Almeida L (1990) Speeding-up backpropagation. In: Eckmiller R (ed) Advanced neural computers. North-Holland, Amsterdam, pp 151–156
Najim K, Chtourou M, Thibault J (1992) Neural network synthesis using learning automata. J Syst Eng 2(4):192–197
Zhang N, Wu W, Zheng G (2006) Convergence of gradient method with momentum for two-layer feedforward neural networks. IEEE Trans Neural Netw 17(2):522–525
Bortoletti A, Di FIore C, Fanelli S, Zellini P (2003) A new class of quasi-newtonian methods for optimal learning in MLP-networks. IEEE Trans Neural Netw 14(2):263–273
Lera G, Pinzolas M (2002) Neighborhood based Levenberg-Marquardt algorithm for neural network training. IEEE Trans Neural Netw 13(5):1200–1203
Abid S, Fnaiech F, Najim M (2001) A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm. IEEE Trans Neural Netw 12(2):424–430
Nickolai SR (2000) The layer-wise method and the backpropagation hybrid approch to learning a feedforward neural network. IEEE Trans Neural Netw 11(2):295–305
Oscar FR, Deniz E, Jose CP (2003) Accelerating the convergence speed of neural networks learning methods using least squares. In: Proceedings of ESANN’2003, Belgium, 23–25 April, pp 255–260
Bilski J (2005) The UD RLS algorithm for training feedforward neural networks. Int J Appl Math Comput Sci 15(1):115–123
Leung CS, Tsoi AC, Chan LW (2001) Two regularizers for recursive least squared algorithm in feedforward multilayered neural networks. IEEE Trans Neural Netw 12(6):1314–1332
Huntsberger TL, Ajjimarangsee P (1989) Parallel self organizing feature maps for unsupervised pattern recognition. Int J Gen Syst 16:357–372
Bezdek JC, Tsao EC, Pal NR (1992) Fuzzy Kohonen clustering networks. In: Proceedings of IEEE international conference on fuzzy systems, March 1992, San Diego, pp 1035–1041
Zweiri YH, Whidborne JF, Seneviratne LD (2003) Three-term backpropagation algorithm. Neurocomputing 50:305–318
Sha D, Bajie BV (2002) An on line hybrid learning algorithm for multilayer perceptron in identification problems. Comput Electr Eng 28:587–598
Liu P, Li H (2004) Efficient learning algorithms for three-layer regular feedforward fuzzy neural networks. IEEE Trans Neural Netw 15(3):545–558
Ben Nasr M, Chtourou M (2006) A hybrid training algorithm for feedforward neural networks. Neural Process Lett 24(2):107–117
Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2:359–366
Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480
Alexander GP, Benito F, Amir FA, Jayakumar M, Wei KT (1994) An accelerated learning algorithm for multilayer networks. IEEE Trans Neural Netw 5(3):493–497
Mackey MC, Glass L (1977) Oscillation and chaos in physiological control system. Science 197:287–289
Box GE, Jenkins GM (1970) Time series analysis, forecasting and control. Holden Day, San Francisco
Kim J (1999) Adaptive neuro-fuzzy inference system and their application to non linear dynamical system. Neural Netw 12:1301–1319
Ben Nasr M, Chtourou M, A constructive hybrid training algorithm for feedforward neural networks. Submitted to Int J Syst Sci
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nasr, M.B., Chtourou, M. A fuzzy neighborhood-based training algorithm for feedforward neural networks. Neural Comput & Applic 18, 127–133 (2009). https://doi.org/10.1007/s00521-007-0165-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-007-0165-z