Abstract
Due to the complex underwater imaging environment and illumination conditions, underwater images have some quality degradation problems, such as low contrast, color distortion, texture blur and uneven illumination, which seriously restrict the application in underwater work. In order to solve these problems, we proposed a multi-scale feature fusion CNN based on underwater imaging model in this paper called Multi-Scale Convolution Underwater Image Restoration Network (MSCUIR-Net). Unlike most previous models that estimated the background light and transmittance, respectively, our model unifies the two parameters into one, predicts the univariate linear physical model through lightweight CNN, and directly generates end-to-end clean images. Based on the underwater imaging model, we synthesized the underwater image training set can simulate the shallow water to deep water environment. Then, we do experiments on synthetic images and real underwater images, and prove the superiority of this method through image evaluation indexes. The experimental results show that MSCUIR-Net has a good effect on underwater image restoration.
Similar content being viewed by others
Notes
The error of a sample defined on a single sample.
The average of all sample errors defined on the whole training set, the average of all loss function values.
In the process of back propagation, the gradient is getting larger and larger, and each layer needs to update a large weight, resulting in the divergence of results.
References
Hegerl, G.C.: Ocean science. warming the world’s oceans. Science 309(5732), 254–255 (2005)
Cai, L., et al.: Multi-AUV collaborative target recognition based on transfer-reinforcement learning. IEEE Access 11, 39273–392114 (2020)
Benoist, N.M.A., et al.: Monitoring mosaic biotopes in a marine conservation zone by autonomous underwater vehicle. Conserv. Biol. 33(5), 1174–11116 (2019)
Shi, P., et al.: A detection and classification approach for underwater dam cracks. Struct. Health Monit 15(5), 541–554 (2016)
Sun, Y., et al.: Deep submergence rescue vehicle docking based on parameter adaptive control with acoustic and visual guidance. Int. J. Adv. Robot. Syst. 17(2), 172911111420919970 (2020)
Ancuti, C., Ancuti, C.O., Haber T., Bekaert P.: Enhancing underwater images and videos by fusion. In: 2012 IEEE conference on Computer vision and pattern recognition. IEEE, pp. 111–1111 (2012)
Ghani, A.S.A., Isa, N.A.M.: Underwater image quality enhancement through integrated color model with rayleigh distribution. Appl. Soft Comput. 27, 219–230 (2015)
Johnson一Roberson, M., Bryson, M., Friedman, A., et al.: High resolution underwater robotic vision based map** and three一dimensional reconstruction for archaeology. J. Field Robot. 34(4), 625–643 (2017)
Ghani, A.S.A., Isa, N.A.M.: Enhancement of low quality underwater image through integrated global and local contrast correction. Appl. Soft Comput. 37, 332–344 (2015)
Mercado, MA., Ishii, K., Ahn, J.: Deep-sea image enhancement using multi-scale retinex with reverse color loss for autonomous underwater vehicles. In: //Proceedings of IEEE conference on oceans, (2017)
Shahrizan, A., Ghani, A., Fakhri A, et al.: Integration of enhanced background filtering and wavelet fusion for high visibility and detection rate of deep sea underwater image of underwater vehicle. In: //Proceedings of international conference on information and communication technology, (2017)
Treibitz, T., Schechner, YY: Instant 3Descatter. In: // Proceedings of IEEE computer society conference on computer vision and pattern recognition, 11161–111611 (2006)
Treibitz, T., Schechner, Y.Y.: Active polarization descattering. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 3115–3399 (2009)
Li, C.Y., Guo, J.C., Cong, R.M., et al.: Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans. Image Process. 25(12), 5664–5677 (2016)
Dai, C.G., Lin, M.X., Wu, X.J., et al.: Single underwater image restoration by decomposing curves of attenuating color. Opt. Laser Technol. (2010). https://doi.org/10.1016/j.optlastec.2019.105947
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)
Ding, X., Wang, Y., Liang, Z., et al. Towards underwater image enhancement using super-resolution convolutional neural networks. In: //international conference on internet multimedia computing and Service. Springer, Singapore, 479–4116 (2017)
Wang, Y., Zhang, J., Cao., Y., et al.: A deep CNN method for underwater image enhancement. In: // IEEE international conference on image processing. IEEE, 2017:13112–13116
Wang, K.Y., Hu, Y., Chen, J., et al.: Underwater image restoration based on a parallel convolutional neural network. Remote Sens. 11(13), 1591 (2019)
Jaffe, J.S.: Computer modeling and the design of optimal underwater imaging systems. J. Ocean. Eng. 15(2), 101–111 (1990)
Xu Yan, Zeng **angbo.: Underwater image restoration based on a priori and inverse filtering of red dark channel (in chinses). Progress in laser and optoelectronics, 20111, 55 (2): 215–222
Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from rgbd images. In: European conference on computer vision, Springer, 2012, pp. 746–760
Li, B., Peng, X., Wang, Z., et al.: AOD-Net: All-in-One Dehazing Network. In: // 2017 IEEE international conference on computer vision (ICCV). IEEE, 2017
Nair, V., Hinton GE.: Rectified linear units improve restricted boltzmann machines. Icml. 2010:1107–1114
Rafferty, J., Shellito, P., Hyman, N.H., et al.: Practice parameters for sigmoid diverticulitis. Dis. Colon Rectum 49(7), 939–944 (2006)
Fan, E.: Extended tanh-function method and its applications to nonlinear equations. Phys. Lett. A 277(4–5), 212–2111 (2000)
Berman, D., Treibitz, T., Avidan, S.: Diving into haze-lines: color restoration of underwater images. In: Proc. british machine vision conference (BMVC), vol. 1, no. 2, 2017
Li, C., Anwar, S., Porikli, F.: Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recogn. 911, 1070311 (2020)
Nayak, GS., Nayak, D.: Back propagation algorithm. (2013)
Ketkar, N.: Stochastic gradient descent, pp. 113–132. Deep learning with Python. Apress, Berkeley, CA (2017)
Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: Dehazenet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 51117–51911 (2016)
He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)
Drews, P.L., Nascimento, E.R., Botelho, S.S., Campos, M.F.M.: Underwater depth estimation and image restoration based on single images. IEEE Comput. Graphics Appl. 36(2), 24–35 (2016)
Li, C.-Y., Guo, J.-C., Cong, R.-M., Pang, Y.-W., Wang, Bo.: Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans. Image Process. 25(12), 5664–5677 (2016)
Panetta, K., Gao, C., Agaian, S.: Human-visual-system-inspired underwater image quality measures. IEEE J. Oceanic Eng. 41(3), 541–551 (2015)
Acknowledgements
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China (No. 51005142), the Innovation Program of Shanghai Municipal Education Commission (No.14YZ010), and the Natural Science Foundation of Shanghai (No. 14ZR1414900, No.19ZR1419300) for providing financial support for this work.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Tang, Z., Li, J., Huang, J. et al. Multi-scale convolution underwater image restoration network. Machine Vision and Applications 33, 85 (2022). https://doi.org/10.1007/s00138-022-01337-3
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00138-022-01337-3