Abstract
Removing undesirable specular highlight from a single input image is of crucial importance to many computer vision and graphics tasks. Existing methods typically remove specular highlight for medical images and specific-object images, however, they cannot handle the images with text. In addition, the impact of specular highlight on text recognition is rarely studied by text detection and recognition community. Therefore, in this paper, we first raise and study the text-aware single image specular highlight removal problem. The core goal is to improve the accuracy of text detection and recognition by removing the highlight from text images. To tackle this challenging problem, we first collect three high-quality datasets with fine-grained annotations, which will be appropriately released to facilitate the relevant research. Then, we design a novel two-stage network, which contains a highlight detection network and a highlight removal network. The output of highlight detection network provides additional information about highlight regions to guide the subsequent highlight removal network. Moreover, we suggest a measurement set including the end-to-end text detection and recognition evaluation and auxiliary visual quality evaluation. Extensive experiments on our collected datasets demonstrate the superior performance of the proposed method.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chineseocr: Ctpn plus densenet plus ctc based chinese ocr. https://github.com/YCG09/chinese_ocr. Accessed 30 Apr 2021
Paddleocr: Awesome multilingual ocr toolkits based on paddlepaddle. https://github.com/PaddlePaddle/PaddleOCR. Accessed 30 Apr 2021
Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)
Arnold, M., Ghosh, A., Ameling, S., Lacey, G.: Automatic segmentation and inpainting of specular highlights for endoscopic imaging. J. Image Video Process. (2010)
Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: ACM SIGGRAPH, pp. 417–424 (2000)
Fleyeh, H.: Shadow and highlight invariant colour segmentation algorithm for traffic signs. In: IEEE Conference on Cybernetics and Intelligent Systems (2006)
Fu, G., Zhang, Q., Song, C., Lin, Q., **ao, C.: Specular highlight removal for real-world images. Comput. Graph. Forum 38(7), 253–263 (2019)
Funke, I., Bodenstedt, S., Riediger, C., Weitz, J., Speidel, S.: Generative adversarial networks for specular highlight removal in endoscopic images. In: Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 10576, pp. 8–16 (2018)
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)
Guo, X., Chen, Z., Li, S., Yang, Y., Yu, J.: Deep eyes: binocular depth-from-focus on focal stack pairs. In: Chinese Conference on Pattern Recognition and Computer Vision, pp. 353–365 (2019)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Proceedings of the European Conference on Computer Vision, pp. 694–711 (2016)
Khanian, M., Boroujerdi, A.S., Breuß, M.: Photometric stereo for strong specular highlights. ar**v preprint ar**v:1709.01357 (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)
Lee, S.T., Yoon, T.H., Kim, K.S., Kim, K.D., Park, W.: Removal of specular reflections in tooth color image by perceptron neural nets. In: International Conference on Signal Processing Systems, vol. 1, pp. V1–285-V1-289 (2010)
Lin, J., El Amine Seddik, M., Tamaazousti, M., Tamaazousti, Y., Bartoli, A.: Deep multi-class adversarial specularity removal. In: Image Analysis, pp. 3–15 (2019)
Long, S., He, X., Yao, C.: Scene text detection and recognition: The deep learning era. Int. J. Comput. Vis. 129(1), 161–184 (2021)
Lucas, S., Panaretos, A., Sosa, L., Tang, A., Wong, S., Young, R.: Icdar 2003 robust reading competitions. In: International Conference on Document Analysis and Recognition, pp. 682–687 (2003)
Meslouhi, O.E., Kardouchi, M., Allali, H., Gadi, T., Benkaddour, Y.A.: Automatic detection and inpainting of specular reflections for colposcopic images. Central Eur. J. Comput. Sci. 1 (2011)
Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: Spectral normalization for generative adversarial networks. In: International Conference on Learning Representations (2018)
Muhammad, S., Dailey, M.N., Farooq, M., Majeed, M.F., Ekpanyapong, M.: Spec-net and spec-cgan: Deep learning models for specularity removal from faces. Image Vis. Comput. 93, 103823 (2020)
Ortiz, F., Torres, F.: A new inpainting method for highlights elimination by colour morphology. In: International Conference on Pattern Recognition and Image Analysis, pp. 368–376 (2005)
Park, J.W., Lee, K.H.: Inpainting highlights using color line projection. IEICE Trans. Inf. Syst. 90(1), 250–257 (2007)
Ren, W., Tian, J., Tang, Y.: Specular reflection separation with color-lines constraint. IEEE Trans. Image Process. 26(5), 2327–2337 (2017)
Shafer, S.A.: Using color to separate reflection components. Color. Res. Appl. 10(4), 210–218 (1985)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)
Son, M., Lee, Y., Chang, H.S.: Toward specular removal from natural images based on statistical reflection models. IEEE Trans. Image Process. (2020)
Tan, P., Lin, S., Quan, L., Shum, H.Y.: Highlight removal by illumination-constrained inpainting. In: IEEE International Conference on Computer Vision, pp. 164–169 (2003)
Tan, R.T., Nishino, K., Ikeuchi, K.: Separating reflection components based on chromaticity and noise analysis. IEEE Trans. Pattern Anal. Mach. Intell. 26(10), 1373–1379 (2004)
Tian, Z., Huang, W., He, T., He, P., Qiao, Y.: Detecting text in natural image with connectionist text proposal network. In: European Conference on Computer Vision, pp. 56–72 (2016)
Wang, T.C., Efros, A.A., Ramamoorthi, R.: Occlusion-aware depth estimation using light-field cameras. In: IEEE International Conference on Computer Vision, pp. 3487–3495 (2015)
Wang, W., Deng, R., Li, L., Xu, X.: Image aesthetic assessment based on perception consistency. In: Chinese Conference on Pattern Recognition and Computer Vision, pp. 303–315 (2019)
Yang, Q., Wang, S., Ahuja, N.: Real-time specular highlight removal using bilateral filtering. In: European Conference on Computer Vision, pp. 87–100 (2010)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision, pp. 2242–2251 (2017)
Acknowledgements
This work was supported by the National Key R&D Program of China (2019YFB2204104), and the National Natural Science Foundation of China (Nos. 6210071649, 62172415 and 61772523).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Hou, S., Wang, C., Quan, W., Jiang, J., Yan, DM. (2021). Text-Aware Single Image Specular Highlight Removal. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13022. Springer, Cham. https://doi.org/10.1007/978-3-030-88013-2_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-88013-2_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88012-5
Online ISBN: 978-3-030-88013-2
eBook Packages: Computer ScienceComputer Science (R0)