SpringerLink
Log in

Medical image fusion based on type-2 fuzzy sets with teaching learning based optimization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The main objective of image fusion for multimodal medical images is to retrieve valuable information by combining multiple images obtained from various sources into a single image suitable for better diagnosis. In general, the visibility of structural details in medical images is difficult to interpret. The vast majority of the best in class image fusing systems are based on non-fuzzy sets, and the fused image so obtained lags with complementary information. Soft computing techniques like fuzzy sets have been applied to enhance the medical images and to extract the visibility features from the images. Fuzzy sets are strong-minded to be more appropriate for medical image processing as more hesitations are considered compared with non-fuzzy sets. Type-2 fuzzy sets are used in this work. Type-2 fuzzy sets are the fuzzy sets for which the membership function is not a single value for every element but an interval. In this paper, a procedure for efficiently fusing multimodal medical images is presented. In the proposed method, images are initially converted into Type-2 fuzzy images. Next, the enhanced images are decomposed into blocks and the blocks are compared using fitness function, contrast visibility (CV). Then, a decision map (DM) is built by taking the decision for each coefficient using the contrast visibility of the respective block. Then, teaching learning based optimization (TLBO) is introduced to optimize combination factors that change under teaching phase, and learner phase of TLBO. Finally, the fused image is achieved using optimal coefficients. Simulations on several pairs of multimodal medical images are performed and matched with the current fusion approaches. The dominance of the proposed technique is presented and is justified. Fused image quality is also verified with various quality metrics, such as peak signal to noise ratio (PSNR), universal quality index (UQI), structural similarity (SSIM), correlation coefficient (CC), entropy (E), spatial frequency (SF), edge information preservation (QAB/F) and standard deviation (SD).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The datasets generated or analyzed during this study are not publicly available due to the author Ph.D (research) thesis submission but are available from the corresponding author on reasonable request.

References

  1. Gopi Krishna E, Tirupal T (2015) Performance of image fusion techniques for satellite images. Int J Technol Res Eng 2(12):3184–3188

    Google Scholar 

  2. Shanker Mishra HO, Bhatnagar S (2014) MRI and CT image fusion based on wavelet transform. Int J Inf Comput Tech 4(1):47–52

    Google Scholar 

  3. Kaplan I, Kolupka E, Morrissey M (1998) MRI-ultrasound image fusion for 125I prostate implant treatment planning. Int J Radiat Oncol Biol Phys 42(1):294

    Google Scholar 

  4. Baum KG, Raerty K, Helguera M, Schmidt E (2007) Investigation of PET/MRI image fusion schemes for enhanced breast cancer diagnosis. In: Proceedings of IEEE seventh symposium conference on nuclear Science (NSS), vol. 5. p. 3774–80

  5. James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. In Fus 19(1):4–19

    Google Scholar 

  6. Azam MA, Khan KB, Salahuddin S et al (2022) A review on multimodal medical image fusion: compendious analysis of medical modalities, multimodal databases, fusion techniques and quality metrics. Comput Biol Med 144:105253. https://doi.org/10.1016/j.compbiomed.2022.105253

    Article  PubMed  Google Scholar 

  7. Dou W, Ruan S, Chen Y, Bloyet D, Constans JM (2007) A framework of fuzzy information fusion for the segmentation of brain tumour tissues on MR images. Image Vis Comput;25(2)

  8. Marshall S, Matsopoulos G, Brunt J (1995) ‘Multiresolution morphological fusion of MR and CT images of the human brain’. Proc. of IEE in vision. Image Signal Process 141:1–5

    Google Scholar 

  9. Liu Y, Chen X., Rabab KW, Jane Wang, Z (2019) Medical image fusion via convolutional sparsity based morphological component analysis, IEEE Signal Process Lett, 26(3)

  10. Ali FE, El-Dokany IM, Saad AA et al (2008) Curvelet fusion of MR and CT images. Progress Electromagn Res C 3:215–224

    Google Scholar 

  11. Mishra HOS, Bhatnagar S (2014) MRI and CT image fusion based on wavelet transform. Int J Inf Commun Technol 4(1):47–52

    Google Scholar 

  12. Lin KP, Yao WJ (1995) A SPECT-CT image fusion technique for diagnosis of head-neck cancer. IEEE Ann Conf Eng Med Biol Soc 1:377–378

    Google Scholar 

  13. Shangli, C, Junmin, H, Zhongwei, L (2008) ‘Medical image of PET/CT weighted fusion based on wavelet transform’. IEEE Int. Conf Bioinf Biomed Eng, pp. 2523–2525

  14. Daneshvar S, Ghassemian H (2010) MRI and PET image fusion by combining IHS and retina-inspired models. In Fus 11(2):114–123

    Google Scholar 

  15. Megalooikonomou V, Kontos D (2007) Medical data fusion for telemedicine. IEEE Eng Med Biol Mag 26(5):36–42

    PubMed  Google Scholar 

  16. Barra V, Boire JY (2001) A general framework for the fusion of anatomical and functional medical images. Neuro Image 13(3):410–424

    CAS  PubMed  Google Scholar 

  17. Holupka E, Kaplan I, Burdette E et al (1996) Ultrasound image fusion for external beam radiotherapy for prostate cancer. Int J Radiat Oncol Biol Phys 35(5):975–984

    CAS  PubMed  Google Scholar 

  18. Hosseini, HG, Alizad, A, Fatemi, M (2007) ‘Integration of Vibro-Acoustography imaging modality with the traditional mammography’, Int J Biomed Imaging, https://doi.org/10.1155/2007/40980

  19. Mitianoudis N, Stathaki T (2007) Pixel-based and region-based image fusion schemes using ICA bases. Inf Fus 8(2):131–142

    Google Scholar 

  20. Naidu VPS, Raol JR (2008) Pixel-level image fusion using wavelets and principal component analysis. Def Sci J 58(3):338–352

    Google Scholar 

  21. Du J, Li W, **ao B, Nawaz Q (2016) Union Laplacian pyramid with multiple features for medical image fusion. Neurocomputing 194:326–339

    Google Scholar 

  22. Bhatnagar G, Jonathan W, Q.M., Liu, Z. (2015) A new contrast based multimodal medical image fusion framework. Neurocomputing 157:143–152

    Google Scholar 

  23. Toet A (1989) Image fusion by a ratio of low-pass pyramid. Pattern Recogn Lett 9(4):245–253

    ADS  Google Scholar 

  24. Toet A (1989) A morphological pyramidal image decomposition. Pattern Recogn Lett 9(4):255–261

    ADS  Google Scholar 

  25. He C, Liu Q, Li H, Wang H (2010) Multimodal medical image fusion based on HIS and PCA. Procedia Eng 7:280–285

    Google Scholar 

  26. Haddadpour M, et al. (2017) ‘PET and MRI image fusion based on combination of 2-D Hilbert transform and IHS method’, Biom J, https://doi.org/10.1016/j.bj.2017.05.002

  27. Haghighat MBA, Aghagolzadeh A, Seyedarabi H (2011) Multi-focus image fusion for visual sensor networks in DCT domain. Comput Electr Eng 37(5):789–797

    Google Scholar 

  28. Shreyamshakumar BK (2012) Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transform. Signal Image Vid Process

  29. Naidu VPS, Divya M, Maha Lakshmi P (2017) Multi-Modal Medical Image Fusion using Multi-Resolution Discrete Sine Transform. Control Data Fus e-J CADFEJL 1(2):13–26

    Google Scholar 

  30. Yang Y, Park DS, Huang S, Rao N (2010) “Medical image fusion via an effective wavelet-based approach,” EURASIP Journal on Advances in Signal Processing, vol. 2010, article ID 579341

  31. Xua X, Wanga Y, Chen S (2016) Medical image fusion using discrete fractional wavelet transform. Biomed Signal Process Control 27:103–111

    Google Scholar 

  32. Chavan, S, Pawar, A, Talbar, S (2016) ‘Multimodality medical image fusion using rotated wavelet transform’, Iyer, B., Nalbalwar, S., Pawade, R., (Eds.) ICCASP/ICMMD-2016, Adv Intell Syst Res’, 137, pp. 627–635

  33. Shahdoosti, HR, Mehrabi, A (2018) ‘Multimodal image fusion using sparse representation classification in Tetrolet domain’, Digit Signal Process, https://doi.org/10.1016/j.dsp.2018.04.002

  34. Das S, Chowdhury M, Kundu MK (2011) Medical image fusion based on ripplet transform type-I. Progress Electromagn Res B 30:355–370

    Google Scholar 

  35. Liu X, Mei W (2017) Huiqian Du.: ‘structure tensor and nonsubsampled shearlet transform based algorithm for CT and MRI image fusion’. Neurocomputing 235:131–139

    Google Scholar 

  36. Srilatha K, Kaviyarasu S (2015) An efficient directive contrast based multi modal medical image fusion under improved NSCT domain. Res J Pharm, Biol Chem Sci 6(5):775

    Google Scholar 

  37. Srivastava, R, Prakash, O, Khare, A (2016) ‘Local energy-based multimodal medical image fusion in curvelet domain’, IET Comput Vis, https://doi.org/10.1049/iet-cvi.2015.0251

  38. Li B, Peng H, Wang J (2021) A novel fusion method based on dynamic threshold neural P systems and nonsubsampled contourlet transform for multi-modality medical images. Signal Process 178:1–13. https://doi.org/10.1016/j.sigpro.2020.107793

    Article  ADS  Google Scholar 

  39. Lia W, **ea Y, Zhoua H, Hanb Y, Zhana K (2018) Structure-aware image fusion. Optik-Int J Light Electron Opt 172:1–11

    ADS  Google Scholar 

  40. Meher B, Agrawal S, Panda R, Abraham A (2018) ‘A survey on region based image fusion methods’, In Fus, https://doi.org/10.1016/j.inffus.2018.07.010

  41. Prakash, O, Park, CM, Khare, A, Jeon, M, Gwak, J (2019) ‘Multiscale fusion of multimodal medical images using lifting scheme based biorthogonal wavelet transform’, Optik, https://doi.org/10.1016/j.ijleo.2018.12.028

  42. Flower J (2005) The redundant discrete wavelet transform and additive noise. IEEE Signal Process Lett 12(9):629–632

    ADS  Google Scholar 

  43. Li X, He M, Roux M (2010) Multifocus image fusion based on redundant wavelet transform. IET Image Process 4(4):283–293

    Google Scholar 

  44. Zadeh LA (1965) Fuzzy Sets. Inform Contr 8(3):338–353

    Google Scholar 

  45. Atanassov KT (1986) Intuitionistic fuzzy sets. Fuzzy Sets Syst 20:87–96

    Google Scholar 

  46. Mendel, JM (2001) Uncertain rule-based fuzzy logic systems introduction and new directions, eaglewood cliffs, NJ: Prentice-Hall

  47. Balasubramaniam P, Ananthi VP (2014) Image fusion using intuitionistic fuzzy sets. In Fus 20:21–30

    Google Scholar 

  48. Tirupal, T., Chandra Mohan, B., Srinivas Kumar, S.: Multimodal medical image fusion based on Yager’s intuitionistic fuzzy sets, Iranian J Fuzzy Syst, vol.16, no.1, February 2019, pp.33–48.

  49. Tirupal T, Chandra Mohan B, Srinivas Kumar S (2017) Multimodal medical image fusion based on Sugeno’s intuitionistic fuzzy sets. ETRI J 39(2):173–180

    Google Scholar 

  50. Manchanda M, Sharma R (2016) A novel method of multimodal medical image fusion using fuzzy transform. J Vis Commun Image Represent 40:197–217

    Google Scholar 

  51. Karnik NN, Mendel JM, Liang Q (1999) Type-2 fuzzy logic systems. IEEE Trans Fuzzy Syst 7:643–658

    Google Scholar 

  52. Rao RV, Savsani VJ, Vakharia DP (2011a) Teaching-learning-based optimization: a novel method for constrained mechanical design optimization problems. Comput Aided Des 43(3):303–315

    Google Scholar 

  53. Vedat T (2012) Design of planar steel frames using Teaching–Learning Based Optimization. Eng Struct 34:225–232

    Google Scholar 

  54. ** H, Wang Y (2014) A fusion method for visible and infrared images based on contrast pyramid with teaching learning based optimization. Infrared Phys Technol 64:134–142

    ADS  Google Scholar 

  55. Mukhopadhyay S, Chanda B (2001) Fusion of 2d gray scale images using multiscale morphology. Pattern Recogn 34:1939–1949

    ADS  Google Scholar 

  56. Bloch I, Colliot O, Camara O et al (2005) Fusion of spatial relationships for guiding recognition, example of brain structure recognition in 3D MRI. Pattern Recogn Lett 26(4):449–457

    ADS  Google Scholar 

  57. Liu Y, Chen X, Ward RK, Wang ZJ (2019) Medical image fusion via convolutional sparsity based morphological component analysis. IEEE Signal Process Lett 26(3):485–489. https://doi.org/10.1109/LSP.2019.2895749

    Article  ADS  Google Scholar 

  58. Li H, Deklerck R, Cuyper BD et al (1995) Object recognition in brain CT-scans: knowledge-based fusion of data from multiple feature extractors. IEEE Trans Med Imaging 14(2):212–229

    CAS  PubMed  Google Scholar 

  59. Rogova GL, Stomper PC (2002) Information fusion approach to microcalcification characterization. In Fus 3(2):91–102

    Google Scholar 

  60. Raza, M, Gondal, I, Green, D, et al. (2005) ‘Classifier fusion to predict breast cancer tumors based on microarray gene expression data’, Knowledge-Based Intelligent Information and Engineering Systems, Springer, pp. 866–874

  61. Qu G, Zhang D, Yan P (2001) Medical image fusion by wavelet transform modulus maxima. Opt Express 9(4):184–190

    Google Scholar 

  62. Kor, S, Tiwary, U (2004) ‘Feature level fusion of multimodal medical images in lifting wavelet transform domain’. IEEE 26th Int. Conf. On EMBS, San Francisco, USA, pp. 1479–1482

  63. Yang L, Guo BL, Ni W (2008) Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neuro Comput 72:203–211

    Google Scholar 

  64. Yang, B, **g, Z (2008) ‘Medical image fusion with a shift-invariant morphological wavelet’, IEEE Cybern Intell Syst, pp. 175–178

  65. Singh, R, Vatsa, M, Noore, A (2009) ‘Multimodal medical image fusion using redundant discrete wavelet transform’. IEEE Int Conf on Advances in Pattern Recognition, pp. 232–235

  66. Kavitha C, Chellamuthu C (2010) Multimodal medical image fusion based on integer wavelet transform and neuro-fuzzy. IEEE Int Conf on Signal Image Process:296–300

  67. Singh R, Khare A (2014) Fusion of multimodal medical images using Daubechies complex wavelet transform – a multiresolution approach. In Fus 19:49–60

    Google Scholar 

  68. Yang, Y, Tong, S, Huang, S, et al. (2014) ‘Log-Gabor energy based multimodal medical image fusion in NSCT domain’, computational and mathematical methods in medicine, Hindawi Publishing Corporation, https://doi.org/10.1155/2014/835481

  69. Qiu, C, Wang, Y, Zhang, H, **a, S (2017) ‘Image fusion of CT and MR with sparse representation in NSST domain’, Comput Math Methods Med, Article ID 9308745, https://doi.org/10.1155/2017/9308745

  70. Vakaimalar, E, Mala, K, Suresh Babu, R (2019) ‘Multifocus image fusion scheme based on discrete cosine transform and spatial frequency’, Multimed Tools Appl, https://doi.org/10.1007/s11042-018-7124-9

  71. Broussard RP, Rogers SK, Oxley ME et al (1999) Physiologically motivated image fusion for object detection using a pulse coupled neural network. IEEE Trans Neural Netw 10(3):554–563

    CAS  PubMed  Google Scholar 

  72. Szu, H, Kopriva, I, Hoekstra, P, et al. (2003) ‘Early tumor detection by multiple infrared unsupervised neural nets fusion’. Proc. of IEEE 25th Annual Int. Conf. on Engineering in Medicine and Biology Society, 2, pp. 1133–1136

  73. El-Hoseny HM, El Kareh ZZ, Mohamed WA et al (2019) An optimal wavelet-based multi-modality medical image fusion approach based on modified central force optimization and histogram matching. Multimed Tools Appl 78:26373–26397. https://doi.org/10.1007/s11042-019-7552-1

    Article  Google Scholar 

  74. Duan J, Mao S, ** J, Zhou Z, Chen L, Chen CLP (2021) A novel GA-based optimized approach for regional multimodal medical image fusion with Superpixel segmentation. IEEE Access 9:96353–96366. https://doi.org/10.1109/ACCESS.2021.3094972

    Article  Google Scholar 

  75. Zuo Q, Zhang J, Yang Y (2021) DMC-fusion: deep multi-Cascade fusion with classifier-based feature synthesis for medical multi-modal images. IEEE J Biomed Health Inform 25(9):3438–3449. https://doi.org/10.1109/JBHI.2021.3083752 Epub 2021 Sep 3

    Article  PubMed  Google Scholar 

  76. Li S, **e Y, Wang G, Zhang L, Zhou W (2022) Adaptive multimodal fusion with attention guided deep supervision net for grading hepatocellular carcinoma. IEEE J Biomed Health Inf 26(8):4123–4131. https://doi.org/10.1109/JBHI.2022.3161466

    Article  Google Scholar 

  77. Wang G, Li W, Gao X, **ao B, Du J (2022, Art no 2508714) Functional and Anatomical Image Fusion Based on Gradient Enhanced Decomposition Model. IEEE Trans Instrum Meas 71:1–14. https://doi.org/10.1109/TIM.2022.3170983

    Article  Google Scholar 

  78. Cai H, Gao Y, Liu M (2023) Graph transformer geometric learning of brain networks using multimodal MR images for brain age estimation. IEEE Trans Med Imaging 42(2):456–466. https://doi.org/10.1109/TMI.2022.3222093

    Article  PubMed  Google Scholar 

  79. Na Y, Lu H, Zhang Y (2008) Content analysis based medical images fusion with fuzzy inference. IEEE Fifth Int Conf on Fuzzy Syst Knowl Discov 3:37–41

    Google Scholar 

  80. Assareh, A, Volkert, LG (2009) ‘Fuzzy rule base classifier fusion for protein mass spectra based ovarian cancer diagnosis’, IEEE Symp Comput Intell Bioinf Comput Biol, pp. 193–199

  81. Teng, J, Wang, S, Zhang, J, et al. (2010) ‘Neuro-fuzzy logic based fusion algorithm of medical images’, IEEE Int. Congress on Image and Signal Processing, pp. 1552–1556

  82. Das, S, Kundu, MK (2013) ‘A neuro-fuzzy approach for medical image fusion’, IEEE Trans Biomed Eng, pp. 1–7

  83. Singh P, Huang Y-P, Chu W-J, Lee J-H (2020) "A Fuzzy-Entropy and Image Fusion Based Multiple Thresholding Method for the Brain Tumor Segmentation," 2020 IEEE international conference on systems, man, and cybernetics (SMC), Toronto, ON, Canada, pp. 2446–2451, https://doi.org/10.1109/SMC42975.2020.9283171

  84. Jana M, Das A (2023) "Multimodal Medical Image Fusion Using Two- Stage Decomposition Technique to Combine the Significant Features of Spatial Fuzzy Plane and Transformed Frequency Plane," in IEEE Trans Instrum Meas, vol. 72, pp. 1-10, Art no 5005910, https://doi.org/10.1109/TIM.2023.3240222

  85. Ensafi P, Tizhoosh HR (2005) Type II fuzzy image enhancement. In: Kamel M, Campilho A (eds) Lecture notes in computer sciences, vol 3656. Springer, Berlin, Germany, pp 159–166

    Google Scholar 

  86. Abdul BS, Arfan Jafar M, Hussain A, Anwar MM (2011) Block based pixel level multi-focus image fusion using particle swarm optimization. Int J Innov Comput Inf Control 7(7(A)):3583–3596

    Google Scholar 

  87. Chaira T (2012) A rank ordered filter for medical image edge enhancement and detection using intuitionistic fuzzy set. Appl Soft Comput 12:1259–1266

    Google Scholar 

  88. Xydeas CS, Petrovic V (2000) Objective image fusion performance measure. Electron Lett 36(4):308–309

    ADS  Google Scholar 

  89. Jagalingam P, Hegde AV (2015) A review of quality metrics for fused image. Aquatic Procedia 4:133–142. https://doi.org/10.1016/j.aqpro.2015.02.019

    Article  Google Scholar 

  90. Srivastava A, Aggarwal AK, “Medical image fusion in spatial and transform domain: a comparative analysis”, Handbook of Research on Advanced Concepts in Real-Time Image and Video Processing,https://doi.org/10.4018/978-1-5225-2848-7.ch011

  91. Aggarwal, AK Fusion and enhancement techniques for processing of multispectral images. In: Avtar, R., Watanabe, T. (eds) unmanned aerial vehicle: applications in agriculture and environment. Springer, Cham https://doi.org/10.1007/978-3-030-27157-2_12

  92. **ao J, Aggarwal AK, Duc NH, Arya A, Rage UK, Avtar R, “A review of remote sensing image spatiotemporal fusion: challenges, applications and recent trends”, Remote Sens Appl Soc Env, https://doi.org/10.1016/j.rsase.2023.101005.

Download references

Funding

No funding was received to assist with this research.

Author information

Authors and Affiliations

  1. Research Scholar, Jawaharlal Nehru Technological University Anantapur, Ananthapuramu, Andhra Pradesh, India

    Kanike Vijay Kumar

  2. Rajeev Gandhi Memorial College of Engineering and Technology, Affiliated to Jawaharlal Nehru Technological University Anantapur, Ananthapuramu, Andhra Pradesh, India

    Anchula Sathish

Authors
  1. Kanike Vijay Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anchula Sathish
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kanike Vijay Kumar.

Ethics declarations

Competing interests

The authors have no competing interest in any material discussed in this research.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, K.V., Sathish, A. Medical image fusion based on type-2 fuzzy sets with teaching learning based optimization. Multimed Tools Appl 83, 33235–33262 (2024). https://doi.org/10.1007/s11042-023-16859-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16859-9

Keywords

Search

Navigation