Abstract
The support vector machine (SVM) has been used as an efficient tool in data mining tasks during the last 2 decades. It is also used for supervised classification with reasonable accuracy. The training time and testing time using an SVM are highly computationally expensive for the classification of large data sets, like multi-band remote sensing images, whether forecasting data, health care data, social welfare data, etc. Due to this problem, researchers have used different training sample reduction methods for the faster classification of more massive data sets with reasonable accuracy. This paper presents a new faster SVM classification method for the remote sensing multi-spectral satellite image that is applied to extract suitable support vectors from the extensive training input by reducing the training time rather than traditional SVM. This method overcomes the classification problem of sizeable remote sensing data sets with better results. We have designed an efficient training sample reduction algorithm to reduce the SVM classifier training time used in the binary classification of large data sets having the number of features two and three, respectively. This binary classification approach has been extended to multi-class classification by identifying valid pairs of classes using a hierarchical clustering algorithm. Experimental findings on different 2D discrete datasets, and remote sensing class datasets performed better than Traditional SVM in terms of training time, classification time, and other performance parameters. The proposed technique is significantly better than traditional SVM for these different datasets. In terms of training time, classification time, and other performance parameters, we have also contrasted our proposed approach with the other three comparable methods.
Similar content being viewed by others
Data availability
Data is available from the authors on reasonable request.
References
Bennett KP, Demiriz A, Shawe-Taylor J (2000) A column generation algorithm for boosting. In: ICML. Citeseer, pp 65–72
Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: Proceedings of the fifth annual workshop on Computational learning theory. ACM, pp 144–152
Burges CJ (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):121–167
Cervantes J, Li X, Yu W et al (2008) Support vector machine classification for large data sets via minimum enclosing ball clustering. Neurocomputing 71(4–6):611–619
Chang CC, Lee YJ (2004) Generating the reduced set by systematic sampling. In: International conference on intelligent data engineering and automated learning. Springer, pp 720–725
Chaudhuri D, Chaudhuri B (1995) A multiseed non-hierarchical clustering technique for data analysis. Int J Syst Sci 26(2):375–385
Chaudhuri D, Chaudhuri BB (1997) A novel multiseed nonhierarchical data clustering technique. IEEE Trans Syst Man Cybern Part B (Cybernetics) 27(5):871–876
Chen PH, Fan RE, Lin CJ (2006) A study on smo-type decomposition methods for support vector machines. IEEE Trans Neural Netw 17(4):893–908
Chien LJ, Chang CC, Lee YJ et al (2010) Variant methods of reduced set selection for reduced support vector machines. J Inf Sci Eng 26(1):183–196
Collobert R, Bengio S (2001) Svmtorch: support vector machines for large-scale regression problems. J Mach Learn Res 1(Feb):143–160
Collobert R, Bengio S, Bengio Y (2002) A parallel mixture of SVMs for very large scale problems. Neural Comput 14(5):1105–1114
Çomak E, Arslan A (2008) A new training method for support vector machines: clustering k-nn support vector machines. Expert Syst Appl 35(3):564–568
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297
Cristianini N, Shawe-Taylor J et al (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge
Folino G, Pizzuti C, Spezzano G (2006) Gp ensembles for large-scale data classification. IEEE Trans Evol Comput 10(5):604–616
Foody GM, Mathur A (2004) A relative evaluation of multiclass image classification by support vector machines. IEEE Trans Geosci Remote Sens 42(6):1335–1343
Foody GM, Mathur A (2004) Toward intelligent training of supervised image classifications: directing training data acquisition for svm classification. Remote Sens Environ 93(1–2):107–117
Friedman J, Hastie T, Tibshirani R (2001) The elements of statistical learning. Springer series in statistics, vol 1. Springer, New York
Gallego FJ (2004) Remote sensing and land cover area estimation. Int J Remote Sens 25(15):3019–3047
Ghoggali N, Melgani F, Bazi Y (2009) A multiobjective genetic svm approach for classification problems with limited training samples. IEEE Trans Geosci Remote Sens 47(6):1707–1718
Gong P (1992) Frequency-based contextual classification and gray-level vector reduction for land-use identification. Photogramm Eng Remote Sens 58:423–437
Jen LR, Lee YJ (2004) Clustering model selection for reduced support vector machines. In: International conference on intelligent data engineering and automated learning. Springer, pp 714–719
Jensen JR, Lulla K (1987) Introductory digital image processing: a remote sensing perspective. In: Introductory digital image processing: a remote sensing perspective. Taylor & Francis
Joachims T (1999) Making large-scale svm learning practical. Advances in kernel methods-support vector learning
Kamel SR, YaghoubZadeh R, Kheirabadi M (2019) Improving the performance of support-vector machine by selecting the best features by gray wolf algorithm to increase the accuracy of diagnosis of breast cancer. J Big Data 6(1):90
Kim SW, Oommen BJ (2004) Enhancing prototype reduction schemes with recursion: a method applicable for “large’’ data sets. IEEE Trans Syst Man Cybern Part B (Cybernetics) 34(3):1384–1397
Kontoes C, Wilkinson G, Burrill A et al (1993) An experimental system for the integration of GIS data in knowledge-based image analysis for remote sensing of agriculture. Int J Geogr Inf Syst 7(3):247–262
Kotsiantis SB, Zaharakis I, Pintelas P (2007) Supervised machine learning: a review of classification techniques. Emerg Artif Intell Appl Comput Eng 160:3–24
Lee YJ, Mangasarian OL (2001a) Rsvm: reduced support vector machines. In: Proceedings of the 2001 SIAM international conference on data mining. SIAM, pp 1–17
Lee YJ, Mangasarian OL (2001b) Ssvm: a smooth support vector machine for classification. Comput Optim Appl 20(1):5–22
Li M, Sethi IK (2006) Confidence-based active learning. IEEE Trans Pattern Anal Mach Intell 28(8):1251–1261
Li X, Cervantes J, Yu W (2007) Two-stage svm classification for large data sets via randomly reducing and recovering training data. In: IEEE international conference on Systems, man and cybernetics, 2007. ISIC. IEEE, pp 3633–3638
Lin KM, Lin CJ (2003) A study on reduced support vector machines. IEEE Trans Neural Netw 14(6):1449–1459
Lin CT, Yeh CM, Liang SF et al (2006) Support-vector-based fuzzy neural network for pattern classification. IEEE Trans Fuzzy Syst 14(1):31–41
Lin WC, Tsai CF, Ke SW et al (2015) Learning to detect representative data for large scale instance selection. J Syst Softw 106:1–8
Liu XZ, Feng GC (2008) Kernel bisecting k-means clustering for svm training sample reduction. In: 19th international conference on pattern recognition, 2008. ICPR 2008. IEEE, pp 1–4
Liu C, Wang W, Wang M et al (2017) An efficient instance selection algorithm to reconstruct training set for support vector machine. Knowl Based Syst 116:58–73
Loosli G, Canu S, Ong CS (2015) Learning svm in krein spaces. IEEE Trans Pattern Anal Mach Intell 38(6):1204–1216
Lu SX, Wang XZ (2004) A comparison among four svm classification methods: Lsvm, nlsvm, ssvm and nsvm. In: Proceedings of 2004 international conference on machine learning and cybernetics (IEEE Cat. No. 04EX826). IEEE, pp 4277–4282
Mangasarian O (1998) Generalized support vector machines. University of Wisconsin-Madison, Tech. rep
Mavroforakis ME, Theodoridis S (2006) A geometric approach to support vector machine (svm) classification. IEEE Trans Neural Netw 17(3):671–682
Mei X, Lee HC, Ky D et al (2020) Artificial intelligence-enabled rapid diagnosis of patients with COVID-19. Nat Med 26(8):1224–1228
Michie D, Spiegelhalter DJ, Taylor C et al (1994) Machine learning, neural and statistical classification
Mitra P, Murthy C, Pal SK (2004) A probabilistic active support vector learning algorithm. IEEE Trans Pattern Anal Mach Intell 26(3):413–418
Musavi MT, Ahmed W, Chan KH et al (1992) On the training of radial basis function classifiers. Neural Netw 5(4):595–603
Nalepa J, Kawulok M (2019) Selecting training sets for support vector machines: a review. Artif Intell Rev 52(2):857–900
Osuna E, Freund R, Girosi F (1997a) An improved training algorithm for support vector machines. In: Neural networks for signal processing VII. Proceedings of the 1997 IEEE signal processing society workshop. IEEE, pp 276–285
Osuna E, Freund R, Girosit F (1997b) Training support vector machines: an application to face detection. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 130–136
Pal M, Mather PM (2003) An assessment of the effectiveness of decision tree methods for land cover classification. Remote Sens Environ 86(4):554–565
Paoli A, Melgani F, Pasolli E (2009) Clustering of hyperspectral images based on multiobjective particle swarm optimization. IEEE Trans Geosci Remote Sens 47(12):4175–4188
Platt JC (1999) 12 fast training of support vector machines using sequential minimal optimization, advances in kernel methods. Support vector learning 185–208
Provost F, Kolluri V (1999) A survey of methods for scaling up inductive algorithms. Data Min Knowl Discov 3(2):131–169
Purnami SW, Zain JM, Embong A (2010a) Data mining technique for medical diagnosis using a new smooth support vector machine. In: International conference on networked digital technologies. Springer, pp 15–27
Purnami SW, Zain JM, Embong A (2010b) A new expert system for diabetes disease diagnosis using modified spline smooth support vector machine. In: International conference on computational science and its applications. Springer, pp 83–92
Purnami SW, Embong A, Zain JM et al (2009) A new smooth support vector machine and its applications in diabetes disease diagnosis. J Comput Sci 5(12):1003–1008
Rifkin R (2000) Svmfu a support vector machine package’in. http://five-percentnation.mit.edu/personalpages/rif.SvmFu/index.html
Schwaighofer A, Tresp V (2001) The bayesian committee support vector machine. In: International conference on artificial neural networks. Springer, pp 411–417
Sharif I, Chaudhuri D (2019) A multiseed-based svm classification technique for training sample reduction. Turk J Electr Eng Comput Sci 27(1):595–604
Shih L, Rennie JD, Chang YH et al (2003) Text bundling: statistics based data-reduction. In: Proceedings of the 20th international conference on machine learning (ICML-03), pp 696–703
Strack R, Kecman V, Strack B et al (2013) Sphere support vector machines for large classification tasks. Neurocomputing 101:59–67
Stuckens J, Coppin P, Bauer M (2000) Integrating contextual information with per-pixel classification for improved land cover classification. Remote Sens Environ 71(3):282–296
Tan K, Du P (2011) Combined multi-kernel support vector machine and wavelet analysis for hyperspectral remote sensing image classification. Chin Opt Lett 9(1):011003
Tong S, Koller D (2001) Support vector machine active learning with applications to text classification. J Mach Learn Res 2(Nov):45–66
Trafalis TB, Ince H (2002) Benders decomposition technique for support vector regression. In: Proceedings of the 2002 international joint conference on neural networks. IJCNN’02 (Cat. No. 02CH37290). IEEE, pp 2767–2772
Tresp V (2000) A bayesian committee machine. Neural Comput 12(11):2719–2741
Tsai CF (2005) Training support vector machines based on stacked generalization for image classification. Neurocomputing 64:497–503
Valdimir V, Vapnik N (1995) The nature of statistical learning theory
Vapnik V (1998a) Statistical learning theory. Wiley, New York
Vapnik VN (1998b) Vlamimirvapnik. “statistical learning theory”, vol 1
Wang L (2005) Support vector machines: theory and applications, vol 177. Springer Science & Business Media, Berlin
Wang L, Jia X (2009) Integration of soft and hard classifications using extended support vector machines. IEEE Geosci Remote Sens Lett 6(3):543–547
Wang D, Shi L (2008) Selecting valuable training samples for svms via data structure analysis. Neurocomputing 71(13–15):2772–2781
Wang J, Neskovic P, Cooper LN (2005) Training data selection for support vector machines. In: International conference on natural computation. Springer, pp 554–564
Yuan Y, Yan J, Cx Xu (2005) Polynomial smooth support vector machine (pssvm). Jisuanji Xuebao (Chin J Comput) 28(1):9–17
Yuan Y, Fan W, Pu D (2007) Spline function smooth support vector machine for classification. J Ind Manag Optim 3(3):529
Yu H, Yang J, Han J (2003) Classifying large data sets using svms with hierarchical clusters. In: Proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 306–315
Zhou F, Lu BL (2007) Learning concepts from large-scale data sets by pairwise coupling with probabilistic outputs. In: International joint conference on neural networks, 2007. IJCNN 2007. IEEE, pp 524–529
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. We are here by declaring that we have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chowdhury, K., Chaudhuri, D. & Pal, A.K. A faster SVM classification technique for remote sensing images using reduced training samples. J Ambient Intell Human Comput 14, 16807–16827 (2023). https://doi.org/10.1007/s12652-023-04689-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-023-04689-4