Abstract
This paper presents a new hybrid classifier that combines the probability based Bayesian Network paradigm with the Nearest Neighbor distance based algorithm. The Bayesian Network structure is obtained from the data by using the K2 structural learning algorithm. The Nearest Neighbor algorithm is used in combination with the Bayesian Network in the deduction phase. For those data bases in which some variables are continuous valued, automatic discretizations of the data are performed. We show the performance of the new proposed approach compared with the Bayesian Network paradigm and with the well known Naive Bayes classifier in some standard databases; the results obtained by the new algorithm are better or equal according to the Wilcoxon statistical test.
This work was supported by the University of the Basque Country and by the Gipuzkoako Foru Aldundi Txit Gorena under OF147/2002 grant.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Aha, D., Kibler, D., Albert, M.: Instance-based learning algorithms. Machine Learning 6, 37–66 (1991)
Castillo, E., Gutiérrez, J., Hadi, A.: Expert Systems and Probabilistic Network Models. Springer, Heidelberg (1997)
Catlett, J.: On chanching continuous attributes into ordered discrete attributes. In: Verlag, S. (ed.) Proceedings of the European Working Session on Learning, pp. 164–178 (1991)
Cooper, G.F., Herskovits, E.A.: A bayesian method for the induction of probabilistic networks from data. Machine Learning 9, 309–347 (1992)
Cowell, R.G., Dawid, A.P., Lauritzen, S., Spiegelharter, D.J.: Probabilistic Networks and Expert Systems. Springer, Heidelberg (1999)
Dasarathy, B.V.: Nearest neighbor (nn) norms: Nn pattern recognition classification techniques. IEEE Computer Society Press, Los Alamitos (1991)
Dietterich, T.G.: Machine learning research: four current directions. AI Magazine 18(4), 97–136 (1997)
Dougherty, J., Kohavi, R., Sahami, M.: Supervised and unsupervised discretization of continuous features. In: International Conference on Machine Learning, pp. 194–202 (1995)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Machine Learning 19(4), 131–163 (1997)
Heckerman, D., Geiger, D., Chickering, D.M.: Learning bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)
Henrion, M.: Propagating uncertainty in bayesian networks by probabilistic logic sampling. In: Proceedings of the Fourth Conference on Uncertainty in Artificial Intelligence, pp. 149–163 (1988)
Ho, T.K., Srihati, S.N.: Decision combination in multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 16, 66–75 (1994)
Inza, I., Larrañaga, P., Etxeberria, R., Sierra, B.: Feature subset selection by bayesian networks based optimization. Artificial Intelligence 123(1-2), 157–184 (2000)
Inza, I., Larrañaga, P., Sierra, B.: Feature subset selection by bayesian networks: a comparison with genetic and sequential algorithms. International Journal of Approximate Reasoning 27(2), 143–164 (2001)
Jensen, F.V.: Introduction to Bayesian networks. University College of London (1996)
Kohavi, R.: Scaling up the accuracy of naive-bayes classifiers: a decision-tree hybrid. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (1996)
Lu, Y.: Knowledge integration in a multiple classifier system. Applied Intelligence 6, 75–86 (1996)
Fayyad, U.M., Irani, K.B.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Kauffman, M. (ed.) Proceedings of the 13th International Joint Conference on Artificial Intelligence, pp. 1022–1029 (1993)
Michie, D., Spiegelhalter, D., Taylor, C. (eds.): Machine learning, neural and statistical classification (1995)
Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)
Murphy, P.M., Aha, D.W.: Uci repository of machine learning databases (1994)
Pearl, J.: Evidential reasoning using stochastic simulation of causal models. Artificial Intelligence 32(2), 245–257 (1987)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, Los Altos (1993)
Sierra, B., Inza, I., Larrañaga, P.: Medical bayesian networks. In: Brause, R., Hanisch, E. (eds.) ISMDA 2000. LNCS, vol. 1933, pp. 4–14. Springer, Heidelberg (2000)
Sierra, B., Larrañaga, P.: Predicting survival in malignant skin melanoma using bayesian networks automatically induced by genetic algorithms. an empirical comparision between different approaches. Artificial Intelligence in Medicine 14, 215–230 (1998)
Sierra, B., Lazkano, E.: Probabilistic-weighted k nearest neighbor algorithm: a new approach for gene expression based classification. In: KES 2002 Proceedings, pp. 932–939. IOS Press, Amsterdam (2002)
Sierra, B., Lazkano, E., Inza, I., Merino, M., Larrañaga, P., Quiroga, J.: Prototype selection and feature subset selection by estimation of distribution algorithms. a case study in the survival of cirrhotic patients treated with tips. Artificial Intelligence in Medicine, pp. 20–29 (2001a)
Sierra, B., Serrano, N., Larrañaga, P., Plasencia, E.J., Inza, I., Jiménez, J.J., Revuelta, P., Mora, M.L.: Using bayesian networks in the construction of a bi-level multi-classifier. Artificial Intelligence in Medicine 22, 233–248 (2001b)
Sierra, B., Serrano, N., Larrañaga, P., Plasencia, E.J., Inza, I., Jiménez, J.J., Revuelta, P., Mora, M.L.: Machine learning inspired approaches to combine standard medical measures at an intensive care unit. In: Horn, W., Shahar, Y., Lindberg, G., Andreassen, S., Wyatt, J.C. (eds.) AIMDM 1999. LNCS (LNAI), vol. 1620, pp. 366–371. Springer, Heidelberg (1999)
Stone, M.: Cross-validation choice and assessment of statistical procedures. Journal Royal of Statistical Society 36, 111–147 (1974)
Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics 1, 80–83 (1945)
Xu, L., Kryzak, A., Suen, C.Y.: Methods for combining multiple classifiers and their applications to handwriting recognition. IEEE Transactionson SMC 22, 418–435 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lazkano, E., Sierra, B. (2003). BAYES-NEAREST: A New Hybrid Classifier Combining Bayesian Network and Distance Based Algorithms. In: Pires, F.M., Abreu, S. (eds) Progress in Artificial Intelligence. EPIA 2003. Lecture Notes in Computer Science(), vol 2902. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24580-3_24
Download citation
DOI: https://doi.org/10.1007/978-3-540-24580-3_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20589-0
Online ISBN: 978-3-540-24580-3
eBook Packages: Springer Book Archive