Abstract
The aim of this study is to evaluate and compare the performances of 5 machine learning (ML) techniques for predicting the spatial probability of landslide at Atsuma, Japan, and Mt. Umyeon, Korea. 5 ML models used are Naïve Bayes (NB), ensemble learning (random forest (RF) and adaboost (AB)), and deep learning (multilayer perceptron (MLP) and convolutional neural network (CNN)) models. Real landslide events at the study areas are randomly separated to the training set for landslide map** and the validation set for assessing performance. To assess the performance of the used models, the resulting models are validated using receiver operating characteristic (ROC) curve. The success rate curves show that the areas under the curve (AUC) for the NB, RF, AB, MLP, and CNN are 85.1, 88.8, 88.6, 87.5, and 95.0%, respectively, at Atsuma and 68.7, 85.6, 90.5, 81.6, and 92.0%, respectively, at Mt. Umyeon. Similarly, the validation results show that the areas under the curve for the NB, RF, AB, MLP, and CNN are 84.3, 87.1, 87.1, 86.7, and 89.7%, respectively, at Atsuma and 64.9, 85.5, 83.9, 84.7, and 90.5%, respectively, at Mt. Umyeon. In addition, statistical tests such as Chi-square test and difference of proportions test show that all classified landslide susceptibility maps have statistical significance and the significant difference in classified landslide susceptibility maps from different ML models. The comparison results among 5 ML models show that the CNN model had the best performance and NB model had the worst performance in both study areas.
Similar content being viewed by others
References
Abella EAC, Van Westen CJ (2007) Generation of a landslide risk index map for Cuba using spatial multi-criteria evaluation. Landslides 4(4):311–325. https://doi.org/10.1007/s10346-007-0087-y
Akgun A (2012) A comparison of landslide susceptibility maps produced by logistic regression, multi-criteria decision, and likelihood ratio methods: a case study at İzmir, Turkey. Landslides 9(1):93–106
Alkhasawneh MS, Ngah UK, Tay LT, Isa M, Ashidi N, Al-Batah MS (2014) Modeling and testing landslide hazard using decision tree. Journal of Applied Mathematics 2014
Althuwaynee OF, Pradhan B, Park H-J, Lee JH (2014) A novel ensemble bivariate statistical evidential belief function with knowledge-based analytical hierarchy process and multivariate statistical logistic regression for landslide susceptibility map**. Catena 114:21–36
Altmann A, Toloşi L, Sander O, Lengauer T (2010) Permutation importance: a corrected feature importance measure. Bioinformatics 26(10):1340–1347
Arabameri A, Saha S, Roy J, Chen W, Blaschke T, Tien Bui D (2020) Landslide susceptibility evaluation and management using different machine learning methods in the Gallicash River Watershed, Iran. Remote Sens 12(3):475
Ayalew L, Yamagishi H (2005) The application of GIS-based logistic regression for landslide susceptibility map** in the Kakuda-Yahiko Mountains, Central Japan. Geomorphology 65(1–2):15–31. https://doi.org/10.1016/j.geomorph.2004.06.010
Baeza C, Lantada N, Amorim S (2016) Statistical and spatial analysis of landslide susceptibility maps with different classification systems. Environ Earth Sci 75(19):1–17
Bai S, Wang J, Thiebes B, Cheng C, Yang Y (2014) Analysis of the relationship of landslide occurrence with rainfall: A case study of Wudu County, China. Arab J Geosci 7(4):1277–1285. https://doi.org/10.1007/s12517-013-0939-9
Barredo J, Benavides A, Hervás J, van Westen CJ (2002) Comparing heuristic landslide hazard assessment techniques using GIS in the Tirajana basin, Gran Canaria Island, Spain. Int J Appl Earth Obs Geoinf 2(1):9–23. https://doi.org/10.1016/s0303-2434(00)85022-9
Breiman L (2001) Random forests. Mach Learn 45(1):5–32
Brownlee J (2016) Deep learning with python: Develop deep learning models on theano and tensorflow using keras. Machine Learning Mastery
Casadei M, Dietrich WE, Miller NL (2003) Testing a model for predicting the timing and location of shallow landslide initiation in soil-mantled landscapes. Earth Surface Processes and Landforms. The Journal of the British Geomorphological Research Group 28(9):925–950
Catani F, Lagomarsino D, Segoni S, Tofani V (2013) Landslide susceptibility estimation by random forests technique: sensitivity and scaling issues. Nat Hazards Earth Syst Sci 13(11):2815–2831
Chen X, Chen W (2020) GIS-based landslide susceptibility assessment using optimized hybrid machine learning methods. Catena 196:104833
Chen W, Li Y (2020) GIS-based evaluation of landslide susceptibility using hybrid computational intelligence models. Catena 195:104777
Chen W, Pourghasemi HR, Zhao Z (2017a) A GIS-based comparative study of Dempster-Shafer, logistic regression and artificial neural network models for landslide susceptibility map**. Geocarto International 32(4):367–385
Chen W, **e X, Peng J, Wang J, Duan Z, Hong H (2017b) GIS-based landslide susceptibility modelling: a comparative assessment of kernel logistic regression, Naïve-Bayes tree, and alternating decision tree models. Geomatics, Natural Hazards and Risk 8(2):950–973
Chen W, **e X, Wang J, Pradhan B, Hong H, Bui DT, Duan Z, Ma J (2017c) A comparative study of logistic model tree, random forest, and classification and regression tree models for spatial prediction of landslide susceptibility. Catena 151:147–160
Chen W, Chen X, Peng J, Panahi M, Lee S (2020a) Landslide susceptibility modeling based on ANFIS with teaching-learning-based optimization and Satin bowerbird optimizer. Geosci Front 12(1):93–107
Chen W, Zhao X, Tsangaratos P, Shahabi H, Ilia I, Xue W, Wang X, Ahmad BB (2020b) Evaluating the usage of tree-based ensemble methods in groundwater spring potential map**. J Hydrol 583:124602
Chung CF, Fabbri AG (2001) Prediction models for landslide hazard zonation using a fuzzy set approach. Geomorphology and Environmental Impact Assessment Balkema, Lisse, The Netherlands, pp 31–47
Conoscenti C, Di Maggio C, Rotigliano E (2008) GIS analysis to assess landslide susceptibility in a fluvial basin of NW Sicily (Italy). Geomorphology 94(3–4):325–339
Constantin M, Bednarik M, Jurchescu MC, Vlaicu M (2011) Landslide susceptibility assessment using the bivariate statistical analysis and the index of entropy in the Sibiciu Basin (Romania). Environ Earth Sci 63(2):397–406
Das I, Stein A, Kerle N, Dadhwal VK (2012) Landslide susceptibility map** along road corridors in the Indian Himalayas using Bayesian logistic regression models. Geomorphology 179:116–125
Ercanoglu M, Gokceoglu C (2004) Use of fuzzy relations to produce landslide susceptibility map of a landslide prone area (West Black Sea Region, Turkey). Eng Geol 75(3–4):229–250
Ermini L, Catani F, Casagli N (2005) Artificial Neural Networks applied to landslide susceptibility assessment. Geomorphology 66(1-4 SPEC. ISS.):327–343. https://doi.org/10.1016/j.geomorph.2004.09.025
Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139
Frye C (2007) About the geometrical interval classification method. Environmental Systems Research Institute, Inc. https://blogs.esri.com/esri/arcgis/2007/10/18/about-thegeometrical-interval-classification-method
Gardner MW, Dorling SR (1998) Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences. Atmos Environ 32(14–15):2627–2636
Girshick R (2015) Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, 1440–1448
Gökceoglu C, Aksoy H (1996) Landslide susceptibility map** of the slopes in the residual soils of the Mengen region (Turkey) by deterministic stability analyses and image processing techniques. Eng Geol 44(1–4):147–161
Gökçeoğlu C, Ercanoğlu M (2001) Heyelan duyarlılık haritalarının hazırlanmasında kullanılan parametrelere ilişkin belirsizlikler. Yerbilimleri Dergisi 22(23):189–206
Guzzetti F (2000) Landslide fatalities and the evaluation of landslide risk in Italy. Eng Geol 58(2):89–107
Hair JF, Black WC, Babin BJ, Anderson RE (2009) Multivariate data analysis, Upper Saddle River, NJ [etc.]. Pearson Prentice Hall, New York, NY: Macmillan, 24, 899
Haykin SS (2009) Neural networks and learning machines/Simon Haykin. Prentice Hall, New York
Igwe O, Mode W, Nnebedum O, Okonkwo I, Oha I (2014) The analysis of rainfall-induced slope failures at Iva Valley area of Enugu State, Nigeria. Environ Earth Sci 71(5):2465–2480. https://doi.org/10.1007/s12665-013-2647-x
Irigaray C, Fernández T, El Hamdouni R, Chacón J (2007) Evaluation and validation of landslide-susceptibility maps obtained by a GIS matrix method: examples from the Betic Cordillera (southern Spain). Nat Hazards 41(1):61–79
Jeong S, Kim Y, Lee JK, Kim J (2015) The 27 July 2011 debris flows at Umyeonsan, Seoul, Korea. Landslides 12(4):799–813. https://doi.org/10.1007/s10346-015-0595-0
Jibson RW, Harp EL, Michael JA (2000) A method for producing digital probabilistic seismic landslide hazard maps. Eng Geol 58(3–4):271–289
Kavzoglu T, Sahin EK, Colkesen I (2015) An assessment of multivariate and bivariate approaches in landslide susceptibility map**: a case study of Duzkoy district. Nat Hazards 76(1):471–496
Keefer DK (1984) Landslides caused by earthquakes. Geol Soc Am Bull 95(4):406–421
Keith TZ (2014) Multiple regression and beyond: An introduction to multiple regression and structural equation modeling. Routledge
Kuncheva LI (2014) Combining pattern classifiers: methods and algorithms. John Wiley & Sons
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436
Lee S, Hong S-M, Jung H-S (2017) A support vector machine for landslide susceptibility map** in Gangwon Province, Korea. Sustainability 9(1):48
Lee S, Hong S-M, Jung H-S (2018) GIS-based groundwater potential map** using artificial neural network and support vector machine models: the case of Boryeong city in Korea. Geocarto International 33(8):847–861
Li Y, Chen W (2020) Landslide susceptibility evaluation using hybrid integration of evidential belief function and machine learning techniques. Water 12(1):113
Listo F d LR, Vieira BC (2012) Map** of risk and susceptibility of shallow-landslide in the city of São Paulo, Brazil. Geomorphology 169:30–44
Mehta DB, Barot PA, Langhnoja SG (2020). Effect of Different Activation Functions on EEG Signal Classification based on Neural Networks. 2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC), 132–135
Meisina C, Scarabelli S (2007) A comparative analysis of terrain stability models for predicting shallow landslides in colluvial soils. Geomorphology 87(3):207–223
Micheletti N, Foresti L, Robert S, Leuenberger M, Pedrazzini A, Jaboyedoff M, Kanevski M (2014) Machine learning feature selection methods for landslide susceptibility map**. Math Geosci 46(1):33–57
Moore ID, Burch GJ (1986) Physical basis of the length-slope factor in the universal soil loss equation 1. Soil Sci Soc Am J 50(5):1294–1298
Moore ID, Grayson RB, Ladson AR (1991) Digital terrain modelling: a review of hydrological, geomorphological, and biological applications. Hydrol Process 5(1):3–30
Mora CS, Vahrson W-G (1994) Macrozonation methodology for landslide hazard determination. Bull Assoc Eng Geol 31(1):49–58
Nguyen VB-Q, Kim Y-T (2020) Rainfall-earthquake-Induced landslide hazard prediction by Monte Carlo simulation: a case study of MT. Umyeon in Korea. KSCE J Civ Eng 24(1):73–86
Nguyen BQV, Lee SR, Kim YT (2020) Spatial probability assessment of landslide considering increases in pore water pressure during rainfall and earthquakes: Case studies at Atsuma and Mt. Umyeon. CATENA, 187, 104317
Nwankpa C, Ijomah W, Gachagan A, Marshall S (2018) Activation functions: Comparison of trends in practice and research for deep learning. Ar**v Preprint Ar**v 1811:03378
Pack, R. T., Tarboton, D. G., & Goodwin, C. N. (1998). The SINMAP Approach to Terrain Stability Map**. 8th Congress of the International Association of Engineering Geology, 8.
Park S, Kim J (2019) Landslide susceptibility map** based on random Forest and boosted regression tree models, and a comparison of their performance. Appl Sci 9(5):942
Peng L, Niu R, Huang B, Wu X, Zhao Y, Ye R (2014) Landslide susceptibility map** based on rough set theory and support vector machines: A case of the Three Gorges area, China. Geomorphology 204:287–301
Perotto-Baldiviezo HL, Thurow TL, Smith CT, Fisher RF, Wu XB (2004) GIS-based spatial analysis and modeling for landslide hazard assessment in steeplands, southern Honduras. Agric Ecosyst Environ 103(1):165–176
Pham BT, Prakash I (2019) A novel hybrid model of bagging-based naïve bayes trees for landslide susceptibility assessment. Bull Eng Geol Environ 78(3):1911–1925
Pham BT, Bui D, Prakash I, Dholakia M (2016) Evaluation of predictive ability of support vector machines and naive Bayes trees methods for spatial prediction of landslides in Uttarakhand state (India) using GIS. J Geom 10:71–79
Pham BT, Bui DT, Pourghasemi HR, Indra P, Dholakia MB (2017a) Landslide susceptibility assesssment in the Uttarakhand area (India) using GIS: a comparison study of prediction capability of naïve bayes, multilayer perceptron neural networks, and functional trees methods. Theor Appl Climatol 128(1–2):255–273
Pham BT, Bui DT, Prakash I, Dholakia MB (2017b) Hybrid integration of Multilayer Perceptron Neural Networks and machine learning ensembles for landslide susceptibility assessment at Himalayan area (India) using GIS. Catena 149:52–63
Pradhan B (2013) A comparative study on the predictive ability of the decision tree, support vector machine and neuro-fuzzy models in landslide susceptibility map** using GIS. Comput Geosci 51:350–365
Pradhan AMS, Kim Y-T (2014) Relative effect method of landslide susceptibility zonation in weathered granite soil: a case study in Deokjeok-ri Creek, South Korea. Nat Hazards 72(2):1189–1217
Pradhan AMS, Kim Y-T (2017) Spatial data analysis and application of evidential belief functions to shallow landslide susceptibility map** at Mt. Umyeon, Seoul. Korea Bulletin of Engineering Geology and the Environment 76(4):1263–1279
Pradhan AMS, Kim YT (2021) Development and evaluation of relative relief based soil thickness model: a comparative study in Hilly Terrain, South Korea. KSCE J Civ Eng 1–13
Pradhan AMS, Kang H-S, Lee J-S, Tarolli P, Kim Y-T (2016) Shallow landslide hazard modeling by incorporating heavy rainfall statistics and quasi-dynamic wetness index: a case study from Korean mountain. Japanese Geotechnical Society Special Publication 2(27):1012–1016. https://doi.org/10.3208/jgssp.kor-01
Pradhan AMS, Kang HS, Lee JS, Kim YT (2019) An ensemble landslide hazard model incorporating rainfall threshold for Mt. Umyeon, South Korea. Bull Eng Geol Environ 78(1):131–146. https://doi.org/10.1007/s10064-017-1055-y
Rathje EM, Bray JD (2000) Nonlinear coupled seismic sliding analysis of earth structures. J Geotech Geoenviron 126(11):1002–1014
Saldivar-Sali A, Einstein HH (2007) A landslide risk rating system for Baguio, Philippines. Eng Geol 91(2–4):85–99
Sameen MI, Pradhan B, Lee S (2020) Application of convolutional neural networks featuring Bayesian optimization for landslide susceptibility assessment. Catena 186:104249
Sarkar S, Kanungo DP (2004) An integrated approach for landslide susceptibility map** using remote sensing and GIS. Photogramm Eng Remote Sens 70(5):617–625
Schicker R, Moon V (2012) Comparison of bivariate and multivariate statistical approaches in landslide susceptibility map** at a regional scale. Geomorphology 161:40–57
Sharif Razavian, A., Azizpour, H., Sullivan, J., & Carlsson, S. (2014). CNN features off-the-shelf: an astounding baseline for recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 806–813
Shin H-C, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298
SI (The Seoul Institute) (2014). Final Report on the Cause of Landslides in Umyeonsan (Mt.)–Complementary Investigation. Report No. 51-6110000-000649-01 (In Korean)
Simoni S, Zanotti F, Bertoldi G, Rigon R (2008) Modelling the probability of occurrence of shallow landslides and channelized debris flows using GEOtop-FS. Hydrological Processes: An International Journal 22(4):532–545
Šimundić A-M (2008) Measures of diagnostic accuracy: basic definitions. Medical and Biological Sciences 22(4):61
Speight JG (1980) The role of topography in controlling throughflow generation: a discussion. Earth Surface Processes 5(2):187–191
Tallarida RJ, Murray RB (1987) Chi-square test. In Manual of pharmacologic calculations, Springer, pp. 140–142
Tarolli P, Borga M, Chang K-T, Chiang S-H (2011) Modeling shallow landsliding susceptibility by incorporating heavy rainfall statistical properties. Geomorphology 133(3–4):199–211
Tien Bui D, Lofman O, Revhaug I, Dick O (2011) Landslide susceptibility analysis in the Hoa Binh province of Vietnam using statistical index and logistic regression. Nat Hazards 59(3):1413
Tien Bui D, Ho T-C, Pradhan B, Pham B-T, Nhu V-H, Revhaug I (2016a) GIS-based modeling of rainfall-induced landslides using data mining-based functional trees classifier with AdaBoost, Bagging, and MultiBoost ensemble frameworks. Environ Earth Sci 75(14):1101
Tien Bui D, Tuan TA, Klempe H, Pradhan B, Revhaug I (2016b) Spatial prediction models for shallow landslide hazards: a comparative assessment of the efficacy of support vector machines, artificial neural networks, kernel logistic regression, and logistic model tree. Landslides 13(2):361–378
Tien Bui D, Tsangaratos P, Nguyen V-T, Van Liem N, Trinh PT (2020) Comparing the prediction performance of a Deep Learning Neural Network model with conventional machine learning models in landslide susceptibility assessment. Catena 188:104426
Tsangaratos P, Ilia I (2016) Comparison of a logistic regression and Naïve Bayes classifier in landslide susceptibility assessments: The influence of models complexity and training dataset size. Catena 145:164–179
van Beek LPH (2002) Assessment of the influence of changes in land use and climate on landslide activity in a Mediterranean environment
Van Dao D, Jaafari A, Bayat M, Mafi-Gholami D, Qi C, Moayedi H, Van Phong T, Ly H-B, Le T-T, Trinh PT (2020) A spatially explicit deep learning neural network model for the prediction of landslide susceptibility. Catena 188:104451
Van Westen CJ (2000) The modeling of landslide hazards using GIS. Surv Geophys 21(2–3):241–255. https://doi.org/10.1023/A:1006794127521
Vasu NN, Lee S-R (2016) A hybrid feature selection algorithm integrating an extreme learning machine for landslide susceptibility modeling of Mt. Woomyeon, South Korea. Geomorphology 263:50–70
Wang L-J, Sawada K, Moriguchi S (2013) Landslide susceptibility analysis with logistic regression model based on FCM sampling strategy. Comput Geosci 57:81–92
Wang Y, Fang Z, Hong H (2019) Comparison of convolutional neural networks for landslide susceptibility map** in Yanshan County, China. Sci Total Environ 666:975–993
Wang Y, Li Y, Song Y, Rong X (2020) The influence of the activation function in a convolution neural network model of facial expression recognition. Appl Sci 10(5):1897
Xu C, Xu X, Dai F, Saraf AK (2012) Comparison of different models for susceptibility map** of earthquake triggered landslides related with the 2008 Wenchuan earthquake in China. Comput Geosci 46:317–329
Yamagishi H, Yamazaki F (2018) Landslides by the 2018 Hokkaido Iburi-Tobu Earthquake on September 6. Landslides 15(12):2521–2524. https://doi.org/10.1007/s10346-018-1092-z
Yang B, Yin K, Lacasse S, Liu Z (2019) Time series analysis and long short-term memory neural network to predict landslide displacement. Landslides 16(4):677–694
Yilmaz I (2010) Comparison of landslide susceptibility map** methodologies for Koyulhisar, Turkey: conditional probability, logistic regression, artificial neural networks, and support vector machine. Environ Earth Sci 61(4):821–836
Youssef AM, Pourghasemi HR, Pourtaghi ZS, Al-Katheeri MM (2016) Landslide susceptibility map** using random forest, boosted regression tree, classification and regression tree, and general linear models and comparison of their performance at Wadi Tayyah Basin, Asir Region, Saudi Arabia. Landslides 13(5):839–856
Zare M, Pourghasemi HR, Vafakhah M, Pradhan B (2013) Landslide susceptibility map** at Vaz Watershed (Iran) using an artificial neural network model: a comparison between multilayer perceptron (MLP) and radial basic function (RBF) algorithms. Arab J Geosci 6(8):2873–2888
Zhang H, Su J (2004) Naive bayesian classifiers for ranking. European Conference on Machine Learning:501–512
Zhao X, Chen W (2020a) GIS-based evaluation of landslide susceptibility models using certainty factors and functional trees-based ensemble techniques. Appl Sci 10(1):16
Zhao X, Chen W (2020b) Optimization of computational intelligence models for landslide susceptibility evaluation. Remote Sens 12(14):2180
Zhou J w, Cui P, Fang H (2013) Dynamic process analysis for the formation of Yangjiagou landslide-dammed lake triggered by the Wenchuan earthquake, China. Landslides 10(3):331–342. https://doi.org/10.1007/s10346-013-0387-3
Acknowledgments
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Korean Ministry of Education (grant no. 2018R1D1A1B07049360), and the Korea Institute of Energy Technology Evaluation and Planning (KETEP) and the Ministry of Trade, Industry & Energy (MOTIE) of the Republic of Korea (No.20201510100020), and the Brain Korea 21 Plus (BK 21 Plus) initiative.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nguyen, BQV., Kim, YT. Landslide spatial probability prediction: a comparative assessment of naïve Bayes, ensemble learning, and deep learning approaches. Bull Eng Geol Environ 80, 4291–4321 (2021). https://doi.org/10.1007/s10064-021-02194-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10064-021-02194-6