Abstract
The eye usually located at the center of the tropical cyclone is connected to the rapid intensification of tropical cyclone and prediction of its track and intensity. The problem of its localization and detection from satellite imagery in deep learning framework is considered here, where the study conducted deals with 35 named tropical cyclones of different categories over the Bay of Bengal and the Arabian Sea of the North Indian Ocean (NIO). Mask region-based convolutional neural network (MRCNN), which is a variant of RCNN with an addition of a mask branch that produces a mask around the detected object, is considered as the backbone architecture. Since deep learning is time-consuming, we propose embedding the concept of granular computing into MRCNN to speed up its learning mechanism. The granulated mask region-based convolutional neural network (G-MRCNN), thus developed, provides better object(s) localization and increases eye detection accuracy, apart from speedy learning. Two novel indices for eye detection, viz, compactness and eye detection index (EDI), are defined incorporating the shape, area, and compactness (circular) of the predicted mask as well as the detection score. The larger the value of EDI, the more circular and compact the shape of the eye region, and the better the prediction. Different types of granulations ranging from regular to arbitrary shapes have been incorporated in G-MRCNN and the prediction accuracy of each model has been compared against a set of testing data as well as during the time of validation using the aforesaid two indices. EDI is seen to reflect well the eye detection performance, as also judged visually. In that sense it is unique. The results reveal that the performance of the G-MRCNN with k-means (k = 5) clustering-based granulation is better than other methods for the detection and localization of the eye of the tropical cyclone over NIO. The prediction skill of the model is then validated with 4 named tropical cyclones of extremely severe, very severe, and severe categories over NIO.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The corresponding author will provide data supporting the findings of this study upon reasonable request.
Code availability
The corresponding author will provide data supporting the findings of this study upon reasonable request.
References
Almahasneh M, Paiement A, **e X, Aboudarham J (2022) MLMT-CNN for object detection and segmentation in multi-layer and multi-spectral images. Mach Vis Appl 33(1). https://doi.org/10.1007/s00138-021-01261-y
Araujo G, Andrade FAA (2022) Post-processing air temperature weather forecast using artificial neural networks with measurements from meteorological stations. Appl Sci (basel) 12(14):7131. https://doi.org/10.3390/app12147131
Chakraborty DB, Pal SK (2018) Neighborhood rough filter and intuitionistic entropy in unsupervised tracking. IEEE Trans Fuzzy Syst 26(4):2188–2200. https://doi.org/10.1109/tfuzz.2017.2768322
Chalmers C, Fergus P, Curbelo Montanez CA, Longmore SN, Wich SA (2021) Video analysis for the detection of animals using convolutional neural networks and consumer-grade drones. J Unmanned Veh Syst 9(2):112–127. https://doi.org/10.1139/juvs-2020-0018
Chaudhuri S, Dutta D, Goswami S, Middey A (2013) Intensity forecast of tropical cyclones over North Indian Ocean using multilayer perceptron model: skill and performance verification. Nat Hazards (dordr) 65(1):97–113. https://doi.org/10.1007/s11069-012-0346-7
Chaudhuri S, Dutta D, Goswami S, Middey A (2015) Track and intensity forecast of tropical cyclones over the North Indian Ocean with multilayer feed forward neural nets: Track and intensity forecast of tropical cyclones over NIO. Meteorol Appl 22(3):563–575. https://doi.org/10.1002/met.1488
Chen W-K (2011) Environmental applications of granular computing and intelligent systems. Intelligent Systems Reference Library. Springer, Berlin Heidelberg, Berlin, Heidelberg, pp 275–301
Chen R, Zhang W, Wang X (2020) Machine learning in tropical cyclone forecast modeling: A review. Atmosphere (basel) 11(7):676. https://doi.org/10.3390/atmos11070676
Dutta D, Pal SK (2023) Prediction and assessment of the impact of COVID-19 lockdown on air quality over Kolkata: a deep transfer learning approach. Environ Monit Assess 195(1):223. https://doi.org/10.1007/s10661-022-10761-x
Dutta D, Pal SK (2022) Interpretation of black box for short-term predictions of pre-monsoon cumulonimbus cloud events over Kolkata. J of Data, Inf and Manag 4(2):167–183. https://doi.org/10.1007/s42488-022-00071-9
Ganivada A, Dutta S, Pal SK (2011) Fuzzy rough granular neural networks, fuzzy granules, and classification. Theor Comput Sci 412(42):5834–5853. https://doi.org/10.1016/j.tcs.2011.05.038
Gao Q, Lim S, Jia X (2018) Hyperspectral image classification using convolutional neural networks and multiple feature learning. Remote Sens (basel) 10(2):299. https://doi.org/10.3390/rs10020299
Ghiasi B, Noori R, Sheikhian H, Zeynolabedin A, Sun Y, Jun C, Hamouda M, Bateni SM, Abolfathi S (2022) Uncertainty quantification of granular computing-neural network model for prediction of pollutant longitudinal dispersion coefficient in aquatic streams. Sci Rep 12(1):4610. https://doi.org/10.1038/s41598-022-08417-4
Giffard-Roisin S, Yang M, Charpiat G, Kumler Bonfanti C, Kégl B, Monteleoni C (2020) Tropical cyclone track forecasting using fused deep learning from aligned reanalysis data. Front Big Data 3:1. https://doi.org/10.3389/fdata.2020.00001
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 27–30 June 2016; Las Vegas, NV, USA, pp 770–778. https://doi.org/10.1109/CVPR.2016.90
He KM, Gkioxari G, Dollar P, Girshick R (2017) Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV). Venice, IEEE, 2017, 2980–2988. https://doi.org/10.1109/iccv.2017.322
Hong S, Kim S, Joh M, Song S-K (2017) GlobeNet: convolutional neural networks for typhoon eye tracking from remote sensing imagery. ar**v [cs.NE]. https://www.arxiv.org/abs/1708.03417
Khamespanah F, Delavar MR, Moradi M, Sheikhian H (2016) A gis-based multi-criteria evaluation framework for uncertainty reduction in earthquake disaster management using granular Computing. Geod Cartogr 42(2):58–68. https://doi.org/10.3846/20296991.2016.1199139
Kim M, Park M-S, Im J, Park S, Lee M-I (2019) Machine learning approaches for detecting tropical cyclone formation using satellite data. Remote Sens 11(10):1195. https://doi.org/10.3390/rs11101195
Kumar A, Singh AK, Tripathi JN, Sateesh M, Singh V (2021) Evaluation of INSAT-3D-derived hydro-estimator and INSAT multi-spectral rain algorithm over tropical cyclones. J Ind Soc Remote Sens 49(7):1633–1650. https://doi.org/10.1007/s12524-021-01332-7
Kumar S, Dube A, Ashrit R, Mitra AK (2023) A machine learning (ML)-based approach to improve tropical cyclone intensity prediction of NCMRWF ensemble prediction system. Pure Appl Geophys 180(1):261–275. https://doi.org/10.1007/s00024-022-03206-6
Kundu S, Pal SK (2015) FGSN: Fuzzy Granular Social Networks – Model and applications. Inf Sci 314:100–117. https://doi.org/10.1016/j.ins.2015.03.065
Lu C, Kong Y, Guan Z (2020) A mask R-CNN model for reidentifying extratropical cyclones based on quasi-supervised thought. Sci Rep 10:15011. https://doi.org/10.1038/s41598-020-71831-z
Ma L-M (2014) Research Progress on China typhoon numerical prediction models and associated major techniques. Prog Geophys 29:1013–1022
Matsuoka D, Nakano M, Sugiyama D, Uchida S (2018) Deep learning approach for detecting tropical cyclones and their precursors in the simulation by a cloud-resolving global nonhydrostatic atmospheric model. Prog Earth Planet Sci 5:80. https://doi.org/10.1186/s40645-018-0245-y
Nair A, Srujan KSSS, Kulkarni SR, Alwadhi K, Jain N, Kodamana H, Sandeep S, John VO (2022) A deep learning framework for the detection of tropical cyclones from satellite images. IEEE Geosci Remote Sens Lett 19:1–5. https://doi.org/10.1109/lgrs.2021.3131638
Narejo S, Jawaid MM, Talpur S, Baloch R, Pasero EGA (2021) Multi-step rainfall forecasting using deep learning approach. PeerJ Comput Sci 7(e514):e514. https://doi.org/10.7717/peerj-cs.514
Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66. https://doi.org/10.1109/tsmc.1979.4310076
Pal SK, Kumar DA (2023) Adaptive granulation-based convolutional neural networks with single pass learning for remote sensing image classification. IEEE J Sel Top Appl Earth Obs Remote Sens 16:57–70. https://doi.org/10.1109/jstars.2022.3223180
Pal SK, Rosenfeld A (1988) Image enhancement and thresholding by optimization of fuzzy compactness. Pattern Recognit Lett 7(2):77–86. https://doi.org/10.1016/0167-8655(88)90122-5
Pal SK, Bhoumik D, Bhunia Chakraborty D (2020) Granulated deep learning and Z-numbers in motion detection and object recognition. Neural Comput Appl 32(21):16533–16548. https://doi.org/10.1007/s00521-019-04200-1
Pal SK, Kumar DA, Meher SK (2024) GrI-CNN: Granulated deep learning model with interpretable architecture for remote sensing image classification. IEEE Trans Geosci Remote Sens 62:1–12. https://doi.org/10.1109/tgrs.2024.3378529
Park M-S, Kim M, Lee M-I, Im J, Park S (2016) Detection of tropical cyclone genesis via quantitative satellite ocean surface wind pattern and intensity analyses using decision trees. Remote Sens Environ 183:205–214. https://doi.org/10.1016/j.rse.2016.06.006
Pramanik A, Pal SK, Maiti J, Mitra P (2022) Granulated RCNN and multi-class deep SORT for multi-object detection and tracking. IEEE Trans Emerg Top Comput Intell 6(1):171–181. https://doi.org/10.1109/tetci.2020.3041019
Ran Y, Wang H, Tian L, Wu J, Li X (2021) Precipitation cloud identification based on faster-RCNN for Doppler weather radar. EURASIP J Wirel Commun Netw 2021:19. https://doi.org/10.1186/s13638-021-01896-5
Roy C, Kovordányi R (2012) Tropical cyclone track forecasting techniques-A review. Atmos Res 104:40–69
Sarvamangala DR, Kulkarni RV (2022) Convolutional neural networks in medical image understanding: a survey. Evol Intell 15(1):1–22. https://doi.org/10.1007/s12065-020-00540-3
Schubert WH, Montgomery MT, Taft RK, Guinn TA, Fulton SR, Kossin JP, Edwards JP (1999) Polygonal eyewalls, asymmetric eye contraction, and potential vorticity mixing in hurricanes. J Atmos Sci 56(9):1197–1223. https://doi.org/10.1175/1520-0469(1999)056%3c1197:peaeca%3e2.0.co;2
Sen D, Pal SK (2010) Gradient histogram: Thresholding in a region of interest for edge detection. Image vis Comput 28(4):677–695. https://doi.org/10.1016/j.imavis.2009.10.010
Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Patt Anal Machine Intell 39(6):1137–1149. https://doi.org/10.1109/tpami.2016.2577031
Sharma S, Zhang M, Anshika GJ, Zhang H, Kota SH (2020) Effect of restricted emissions during COVID-19 on air quality in India. Sci Total Environ 728(138878):138878. https://doi.org/10.1016/j.scitotenv.2020.138878
Soares EA, Camargo HA, Camargo SJ, Leite DF (2018) Incremental gaussian granular fuzzy modeling applied to hurricane track forecasting. In: IEEE International conference on fuzzy systems. IEEE, pp 1–8
Tian G, Liu L, Ri J, Liu Y, Sun Y (2019) Object Fusion: an object detection and segmentation framework with RGB-D SLAM and convolutional neural networks. Neuro Computing 345:3–14. https://doi.org/10.1016/j.neucom.2019.01.088
Veldkamp S, Whan K, Dirksen S, Schmeits M (2021) Statistical postprocessing of wind speed forecasts using convolutional neural networks. Mon Weather Rev 149(4):1141–1152. https://doi.org/10.1175/mwr-d-20-0219.1
Wang T, Xuan P, Liu Z, Zhang T (2020a) Assistant diagnosis with Chinese electronic medical records based on CNN and BiLSTM with phrase-level and word-level attentions. BMC Bioinform 21:230. https://doi.org/10.1186/s12859-020-03554-x
Wang C, Xu Q, Li X, Cheng Y (2020b) CNN-based tropical cyclone track forecasting from satellite infrared images. In: IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium. Waikoloa, HI, USA, pp 5811–5814. https://doi.org/10.1109/IGARSS39084.2020.9324408
Xu W, Balaguru K, August A, Lalo N, Hodas N, Demaria M, Judi D (2021) Deep Learning Experiments for Tropical Cyclone Intensity Forecasts. Weather Forecast 36:1453–1470
Xu X-Y, Shao M, Chen P-L, Wang Q-G (2022) Tropical cyclone intensity prediction using deep convolutional neural network. Atmosphere (basel) 13(5):783. https://doi.org/10.3390/atmos13050783
Yim J, Ju J, Jung H, Kim J (2015) Image classification using convolutional neural networks with multi-stage feature. Advances in Intelligent Systems and Computing. Springer International Publishing, Cham, pp 587–594
Zadeh LA (1997) Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic. Fuzzy Sets and Systems 90(2):111–127. https://doi.org/10.1016/s0165-0114(97)00077-8
Zaki MJ, Wagner Meira J (2020) Data mining and machine learning: fundamental concepts and algorithms, 2nd edn, Cambridge University Press, March 2020
Zhang P, Cao W, Li W (2021) Surface and high-altitude combined rainfall forecasting using convolutional neural network. Peer Peer Netw Appl 14(3):1765–1777. https://doi.org/10.1007/s12083-020-00938-x
Acknowledgements
The work was done when Prof. S.K. Pal held a National Science Chair, SERB-DST, Govt. of India.
Funding
SERB-National Science Chair, Govt. of India, awarded to Prof. Sankar K. Pal.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Yes.
Consent to participate
Not applicable.
Competing interests
The authors declare that they have no competing interests.
interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Pal, S.K., Biswas, S. & Dutta, D. Granulated mask RCNN and eye detection index (EDI) for detection and localization of eye of tropical cyclone from satellite imagery. J. of Data, Inf. and Manag. (2024). https://doi.org/10.1007/s42488-024-00128-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s42488-024-00128-x