Abstract
In the fields of pattern recognition and image processing, deep learning (DL) technology has been intensively studied. According to the characteristics of multi-modal medical images, medical diagnostic technology, and practical implementation, according to the practical requirements for medical diagnosis, a multi-mode medical image fusion with DL will be suggested. In this study, we present a straightforward but efficient deep learning medical image fusion technique based on convolution neural network with three input images fused by three distinct conventional methods to make use of the gradient information to achieve the multi-exposure image composition in various medical image techniques. The proposed technique is capable of creating a pleasing tone mapped-like high resolution image from many input images utilizing various existing methods by seamlessly compositing them under the direction of gradient-based quality assessment. Particularly, this technique was created based on observations of gradient changes between various exposures and is utilized to improve the fused image instead of other conventional techniques. The usefulness of the suggested approach is demonstrated through experiments with various medical imaging techniques.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12596-023-01123-y/MediaObjects/12596_2023_1123_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12596-023-01123-y/MediaObjects/12596_2023_1123_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12596-023-01123-y/MediaObjects/12596_2023_1123_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12596-023-01123-y/MediaObjects/12596_2023_1123_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12596-023-01123-y/MediaObjects/12596_2023_1123_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12596-023-01123-y/MediaObjects/12596_2023_1123_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12596-023-01123-y/MediaObjects/12596_2023_1123_Fig7_HTML.png)
Similar content being viewed by others
Data availability
All data are available upon request from the corresponding author.
References
W. El-Shafai, I. Almomani, A. Alkhayer, Optical bit-plane-based 3D-JST cryptography algorithm with cascaded 2D-FrFT encryption for efficient and secure HEVC communication. IEEE Access 2(9), 35004–35026 (2021)
A. Alarifi, M. Amoon, M.H. Aly, W. El-Shafai, Optical PTFT asymmetric cryptosystem-based secure and efficient cancelable biometric recognition system. IEEE Access 8, 221246–221268 (2020)
O. Faragallah, A. Afifi, H. El-Sayed, M. Alzain, J. Al-Amri, F. Abd El-Samie, W. El-Shafai, W., Efficient HEVC integrity verification scheme for multimedia cybersecurity applications. IEEE Access 7(8), 167069–167089 (2020)
I. Elashry, W. El-Shafai, E. Hasan, S. El-Rabaie, A. Abbas, A. El-Samie, O. Faragallah, Efficient chaotic-based image cryptosystem with different modes of operation. Multimed. Tools Appl. 79(29), 20665–20687 (2020)
N. Soliman, M. Khalil, A. Algarni, S. Ismail, R. Marzouk, W. El-Shafai, Efficient HEVC steganography approach based on audio compression and encryption in QFFT domain for secure multimedia communication. Multimed. Tools Appl. 80(3), 4789–4823 (2021)
W. El-Shafai, Joint adaptive pre-processing resilience and post-processing concealment schemes for 3D video transmission. 3D Res. 6, 1–13 (2015)
K. Abdelwahab, A. El-atty, M. Saied, W. El-Shafai, S. El-Rabaie, A. El-Samie, Efficient SVD-based audio watermarking technique in FRT domain. Multimed. Tools Appl. 79(9), 5617–5648 (2020)
A. Algarni, G. El Banby, S. Ismail, W. El-Shafai, F. El-Samie, N.F. Soliman, Discrete transforms and matrix rotation based cancelable face and fingerprint recognition for biometric security applications. Entropy 22(12), 1361 (2020)
N. El-Hag, A. Sedik, W. El-Shafai, H. El-Hoseny, A. Khalaf, A. El-Fishawy, G. El-Banby, Classification of retinal images based on convolutional neural network. Microsc. Res. Tech. 84(3), 394–414 (2021)
L. Abou Elazm, S. Ibrahim, M. Egila, H. Shawky, M. Elsaid, W. El-Shafai, F. Abd El-Samie, Cancellable face and fingerprint recognition based on the 3D jigsaw transform and optical encryption. Multimed. Tools Appl. 7(9), 14053–14078 (2020)
W. El-Shafai, S. El-Rabaie, M. El-Halawany, F. El-Samie, Enhancement of wireless 3d video communication using color-plus-depth error restoration algorithms and Bayesian Kalman filtering. Wireless Pers. Commun. 97(1), 245–268 (2017)
O. Faragallah, W. El-Shafai, A. Sallam, I, Elashry, E. EL-Rabaie, A. Afifi, H. El-sayed, Cybersecurity framework of hybrid watermarking and selective encryption for secure HEVC communication. J. Ambient Intell. Humaniz. Comput. 13(2), 1215–1239 (2022)
I. Badr, A. Radwan, E. El-Rabaie, L. Said, G. El Banby, W. El-Shafai, F. Abd El-Samie, Cancellable face recognition based on fractional-order Lorenz chaotic system and Haar wavelet fusion. Digit. Signal Process. 11(6), 103103 (2021)
W. El-Shafai, F. Mohamed, H. Elkamchouchi, M. Abd-Elnaby, A. Elshafee, Efficient and secure cancelable biometric authentication framework based on genetic encryption algorithm. IEEE Access 2(9), 77675–77692 (2021)
F. Abd El-Samie, R. Nassar, M. Safan, M. Abdelhamed, A. Khalaf, G. El Banby, W. El-Shafai, Efficient implementation of optical scanning holography in cancelable biometrics. Appl. Opt. 60(13), 3659–3667 (2021)
S. Ibrahim, M. Egila, H. Shawkey, M. Elsaid, W. El-Shafai, F. Abd El-Samie, Hardware implementation of cancellable biometric systems. In: Fourth IEEE International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), 1145–1152, (2020)
H. El-Hameed, N. Ramadan, W. El-Shafai, A. Khalaf, H. Ahmed, S. Elkhamy, F. El-Samie, Cancelable biometric security system based on advanced chaotic maps. Vis. Comput. 38(6), 2171–2187 (2021)
I. Almomani, W. El-Shafai, A. AlKhayer, A. Alsumayt, S. Aljameel, Proposed biometric security system based on deep learning and chaos algorithms. Comput. Mater. Contin. 74(2), 3515–3537 (2023)
L. Elazm, W. El-Shafai, S. Ibrahim, M. Egila, H. Shawkey, Efficient hardware design of a secure cancellable biometric cryptosystem. Intell. Autom. Soft Comput. 36(1), 929–955 (2023)
W. El-Shafai, M. Elsayed, M. Rashwan, M. Dessouky, A. El-Fishawy, Optical ciphering scheme for cancellable speaker identification system. Comput. Syst. Sci. Eng. 45(1), 563–578 (2023)
A. Ayoup, A. Khalaf, W. El-Shafai, F. Abd El-Samie, F. Alraddady, S. Eldin, Cancellable multi-biometric template generation based on Arnold cat map and aliasing. CMC-Comput. Mater. Contin. 72(2), 3687–3703 (2022)
S. El-Gazar, W. El Shafai, G. El Banby, H. Hamed, G. Salama, M. Abd-Elnaby, F. Abd El-Samie, Cancelable Speaker identification system based on optical-like encryption algorithms. Comput. Syst. Sci. Eng. 43(1), 87–102 (2022)
A. Ayoup, A. Khalaf, F. Alraddady, F. Abd El-Samie, W. El-Shafai, Cancelable multi-biometric template generation based on dual-tree complex wavelet transform. Intell. Autom. Soft Comput. 33(2), 1289–1304 (2022)
I. Almomani, A. AlKhayer, W. El-Shafai, Novel ransomware hiding model using HEVC steganography approach. CMC Comput. Mater. Contin. 70(2), 1209–1228 (2021)
A. Alarifi, S. Sankar, T. Altameem, K. Jithin, M. Amoon, W. El-Shafai, A novel hybrid cryptosystem for secure streaming of high efficiency H. 265 compressed videos in IoT multimedia applications. IEEE Access 8(2), 128548–128573 (2020)
O. Faragallah, H. El-Hoseny, W. El-Shafai, W. El-Rahman, H. El-sayed, S. El-Rabaie, G. Geweid, Optimized multimodal medical image fusion framework using multi-scale geometric and multi-resolution geometric analysis. Multimed. Tools Appl. 81(10), 14379–14401 (2022)
R. Ali, F. El-Sayed, W. El-Shafai, T. Elsayed, Efficient fusion of medical images based on CNN. Menoufia J. Electron. Eng. Res. 30(2), 79–83 (2021)
H.M. El-Hoseny, W.A. El-Rahman, W. El-Shafai, E.S.M. El-Rabaie, K.R. Mahmoud, F.E. Abd El-Samie, O.S. Faragallah, Optimal multi-scale geometric fusion based on non-subsampled contourlet transform and modified central force optimization. Int. J. Imaging Syst. Technol. 29(1), 4–18 (2019)
H. El-Hoseny, W. El-Rahman, W. El-Shafai, G. El-Banby, S. El-Rabaie, F. Abd El-Samie, K. Mahmoud, Efficient multi-scale non-sub-sampled shearlet fusion system based on modified central force optimization and contrast enhancement. Infrared Phys. Technol. 102, 102975 (2019)
W. El-Shafai, S. El-Rabaie, M. El-Halawany, F. Abd El-Samie, Security of 3D-HEVC transmission based on fusion and watermarking techniques. Multimed. Tools Appl. 78(19), 27211–27244 (2019)
W. El-Shafai, N. El-Hag, A. Sedik, G. Elbanby, F. Abd El-Samie, An efficient medical image deep fusion model based on convolutional neural networks. Comput. Mater. Contin. 74(2), 2905–2925 (2023)
W. El-Shafai, F. Khallaf, E. El-Rabaie, F. El-Samie, Robust medical image encryption based on DNA-chaos cryptosystem for secure telemedicine and healthcare applications. J. Ambient. Intell. Humaniz. Comput. 12(10), 9007–9035 (2021)
W. El-Shafai, E. Hemdan, Robust and efficient multi-level security framework for color medical images in telehealthcare services. J. Ambient. Intell. Humaniz. Comput. 8(5), 1–16 (2021)
W. El-Shafai, M. Aly, A. Algarni, F.E. Abd El-Samie, N.F. Soliman, Secure and robust optical multi-stage medical image cryptosystem. CMC-Comput. Mater. Contin. 70(1), 895–913 (2022)
N. Soliman, N. Ali, M. Aly, A. Algarni, W. El-Shafai, F. Abd El-Samie, An efficient breast cancer detection framework for medical diagnosis applications. CMC-Comput. Mater. Contin. 70(1), 1315–1334 (2022)
F. Alqahtani, M. Amoon, W. El-Shafai, A fractional fourier based medical image authentication approach. CMC-Comput. Mater. Contin. 70(2), 3133–3150 (2022)
W. El-Shafai, S. El-Nabi, S. El-Rabaie, A. Ali, N. Soliman, Efficient deep-learning-based autoencoder denoising approach for medical image diagnosis. CMC-Comput. Mater. Contin. 70(3), 6107–6125 (2022)
A. Siam, M. Almaiah, A. Al-Zahrani, A. Elazm, G. El Banby, W. El-Shafai, N. El-Bahnasawy, Secure health monitoring communication systems based on IoT and cloud computing for medical emergency applications. Comput. Intell. Neurosci. 8(3), 1–20 (2021)
E. Hemdan, W. El-Shafai, A. Sayed, CR19: a framework for preliminary detection of COVID-19 in cough audio signals using machine learning algorithms for automated medical diagnosis applications. J. Ambient. Intell. Humaniz. Comput. 3(6), 1–13 (2022)
M. Kaur, D. Singh, Fusion of medical images using deep belief networks. Clust. Comput. 23(2), 1439–1453 (2020)
W. El-Shafai, S. El-Rabaie, M. El-Halawany, F. El-Samie, Recursive Bayesian filtering-based error concealment scheme for 3D video communication over severely lossy wireless channels. Circuits Syst. Signal Process. 37(11), 4810–4841 (2018)
C. Ghandour, W. El-Shafai, and S. El-Rabaie, Medical Image Fusion Based on Weighted Least Square Optimization and Deep Learning Algorithm. In: 2021 9th International Japan-Africa Conference on Electronics, Communications, and Computations (JAC-ECC) IEEE. pp. 159–163, (2021)
O. Faragallah, H. El-Hoseny, W. El-Shafai, W. Abd El-Rahman, H. El-Sayed, S. El-Rabaie, G. Geweid, A comprehensive survey analysis for present solutions of medical image fusion and future directions. IEEE Access 9, 11358–11371 (2020)
N. El-Hag, A. Sedik, G. El-Banby, W. El-Shafai, A. Khalaf, W. Al-Nuaimy, H. El-Hoseny, Utilization of image interpolation and fusion in brain tumor segmentation. Int. J. Num. Methods Biomed. Eng. 37(8), e3449 (2021)
C. Ghandour, W. El-Shafai, and S. El-Rabaie, Comparative Study between Different Image Fusion Techniques Applied on Biomedical Images. In: 2021 9th International Japan-Africa Conference on Electronics, Communications, and Computations (JAC-ECC) IEEE. pp. 165–176, (2021)
A.M. Ayoup, A.A. Khalaf, F. Alraddady, F.E.A. El-Samie, W. El-Safai, S.M.S. Eldin, Selective cancellable multi-biometric template generation scheme based on multi-exposure feature fusion. Intell. Autom. Soft Comput. 33(1), 549–565 (2022)
Acknowledgements
Not applicable.
Funding
The authors did not receive support from any organization for the submitted work.
Author information
Authors and Affiliations
Contributions
All authors are equally contributed.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Consent for publication
All authors are accepting to submit and publish the submitted work.
Ethics approval
All authors are contributing and accepting to submit the current work.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
El-Shafai, W., Ghandour, C. & El-Rabaie, S. Improving traditional method used for medical image fusion by deep learning approach-based convolution neural network. J Opt 52, 2253–2263 (2023). https://doi.org/10.1007/s12596-023-01123-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12596-023-01123-y