SpringerLink
Log in

Evaluation of visual saliency analysis algorithms in noisy images

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Most existing visual saliency analysis algorithms assume that the input image is clean and does not have any disturbances. However, this situation is not always the case. In this paper, we provide an extensive evaluation of visual saliency analysis algorithms in noisy images. We analyze the noise immunity of saliency analysis algorithms by evaluating the performances of the algorithms in noisy images with increasing noise scales and by studying the effects of applying different denoising methods before performing saliency analysis. We use 10 state-of-the-art saliency analysis algorithms and 7 typical image denoising methods on 4 eye fixation datasets and 2 salient object detection datasets. Our experiments show that the performances of saliency analysis algorithms decrease with increasing image noise scales in general. An exception is that the nonlinear features (NF) integrated algorithm shows good noise immunity. We also find that image denoising methods can greatly improve the noise immunity of the algorithms. Our results show that the combination of NF and Median denoising method works best on eye fixation datasets and the combination of saliency optimization (SO) and color block-matching and 3D filtering (C-BM3D) method works best on salient object detection datasets. The combination of SO and Average denoising method works best for applications wherein time efficiency is a major concern for both types of datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Niu, Y., Liu, F., Li, X., Gleicher, M.:Warp propagation for video resizing, CVPR: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 537–544 (2010)

  2. Niu, Y., Liu, F., Li, X., Gleicher, M.: Image resizing via non-homogeneous war**. Multimedia Tools Appl. 56(3), 485–508 (2012)

    Article  Google Scholar 

  3. Niu, Y., Liu, F., Feng, W.C., **, H.: Aesthetics-based stereoscopic photo crop** for heterogeneous displays. IEEE Trans. Multimedia 14(3), 783–796 (2012)

    Article  Google Scholar 

  4. Gao, D., Han, S., Vasconcelos, N.: Discriminant saliency, the detection of suspicious coincidences, and applications to visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(6), 989–1005 (2009)

    Article  Google Scholar 

  5. Liang Z., Fu H., Chi Z., Feng, D.: Image pre-classification based on saliency map for image retrieval. In: ICICS: The 7th International Conference on Information Communications and Signal Processing, IEEE, Macau, pp. 1–5 (2009)

  6. Hadizadeh, H., Bajic, I.: Saliency-aware video compression. IEEE Trans. Image Process. 23(1), 19–33 (2014)

    Article  MathSciNet  Google Scholar 

  7. Li, W., Chang, H., Lien, K., Chang, H., Wang, Y.: Exploring visual and motion saliency for automatic video object extraction spatio-temporal attention modeling procession. IEEE Trans. Image Process. 19(1), 185C198 (2010)

    Google Scholar 

  8. Achanta, R., Estrada, F., Wils, P., Susstrunk, S.: Salient region detection and segmentation. In: international conference on computer vision systems (ICVS ’08), vol. 5008, Springer Lecture Notes in Computer science, pp. 66–75 (2008)

  9. Fu, Y., Cheng, J. Li, Z. and Lu, H.: Saliency cuts: an automatic approach to object segmentation. CVPR: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1–4 (2008)

  10. Zhang, L., Zhang, D., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)

    Article  MathSciNet  Google Scholar 

  11. Zhang, L., Shen, Y., Li, H.: VSI: a visual saliency-induced index for perceptual image quality assessment. IEEE Trans. Image Process. 23(10), 4270–4281 (2014)

    Article  MathSciNet  Google Scholar 

  12. Gide, M.S., Karam, L.J.: Comparative evaluation of visual saliency models for quality assessment task. International Workshop on Video Processing and Quality Metrics for Consumer Electronics, VPQM (2012)

  13. Mittal, A. Moorthy, A.K., Bovik, A.C., Cormack, L.K.: Automatic prediction of saliency on JPEG distorted images. The 3rd International Workshop on Quality of Multimedia Experience (2011)

  14. Kim, C., Milanfar, P.: Visual saliency in noisy images. J. Vis. 13(4), 5 (2013)

    Article  Google Scholar 

  15. Erdem, E., Erdem, A.: Visual saliency estimation by nonlinearly integrating features using region covariances. J. Vis. 13(4), 11,1-20 (2013)

    Article  MathSciNet  Google Scholar 

  16. Hou, X., Zhang, L.: Dynamic visual attention: searching for coding length increments. Adv. Neural Inf. Process. Syst. 681–688 (2009)

  17. Zhu, W., Liang, S., Wei, Y. and Sun, J.: Saliency optimization from robust background detection. CVPR: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 2814–2821 (2014)

  18. Jiang, B., Zhang, L., Lu, H., Yang, C., Yang, M.H.: Saliency detection via absorbing markov chain. In: ICCV International Conferenceon on Computer Vision. IEEE, pp. 1665–1672 (2013)

  19. Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.H.: Saliency detection via graph-based manifold ranking. CVPR: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 3166–3173 (2013)

  20. Wei, Y., Wen, F., Zhu, W. and Sun, J.: Geodesic saliency using background priors. In: ECCV: European Conference on Computer Vision, pp. 29–42 (2012)

  21. Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. CVPR: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 1155–1162 (2013)

  22. Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. CVPR: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 409–416 (2011)

  23. Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. CVPR: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 1597–1604 (2009)

  24. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising with block-matching and 3D filtering. SPIE, Proc (2006)

  25. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)

    Article  MathSciNet  Google Scholar 

  26. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. IEEE International Conference on Computer Vision. pp. 839–846 (1998)

  27. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Color image denoising via sparse 3D collaborative filtering with grou** constraint in luminance-chrominance space. IEEE ICIP (2007)

  28. Kong, X., Li, K., Yang, Q., Wenyin, L. and Yang, M.H.: A new image quality metric for image auto-denoising. In: ICCV: IEEE International Conference on Computer Vision, pp. 2888–2895 (2013)

  29. Burger, H.C., Schuler, C.J., Harmeling, S.: Image denoising: Can plain neural networks compete with BM3D? CVPR: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 2392–2399 (2012)

  30. Borji, A., Sihite, D.N., Itti, L.: Salient object detection: a benchmark. J. In: ECCV: European Conference on Computer Vision, pp. 414–429 (2012)

  31. Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  32. Jain, P., Tyagi, V.: A survey of edge-preserving image denoising methods. Information Systems Frontiers, pp. 1–12 (2014)

  33. Le Meur, O., Le Callet, P., Barba, D., Thoreau, D.: A coherent computational approach to model bottom–up visual attention. IEEE Trans. Pattern Anal. Mach. Intell. 28(5), 802–817 (2006)

    Article  Google Scholar 

  34. Wang, J., Chandler, D.M., Le Callet, P.: Quantifying the relationship between visual salience and visual importance. Spie Human and Electronic imaging. (HVEI) XV, San Jose (2010)

  35. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings 8th Intl Conf. Computer Vision. vol. 2, pp. 416—423 (2001)

  36. Bruce, N.D., Tsotsos, J.K.: Saliency, attention, and visual search: an information theoretic approach. J. Vis. 9(3), 5 (2009)

    Article  Google Scholar 

  37. Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. IEEE International Conference on Computer Vision. pp. 2106–2113 (2009)

  38. Zou, W., Kpalma, K., Liu, Z. and Ronsin, J.: Segmentation driven low-rank matrix recovery for saliency detection. In: BMVC: The 24th British machine vision conference, pp. 1–13 (2013)

Download references

Acknowledgments

This work is partly supported by the National Natural Science Foundation of China under Grant No. 61300102 and No. 61103175, the Fujian Natural Science Funds for Distinguished Young Scholar under Grant No. 2015J06014 and No. 2014J06017, the Natural Science Foundation of Fujian Province under Grant No. 2014J01233, the Fujian Province High School Science Fund for Distinguished Young Scholars under Grant No. JA12016, and the Program for New Century Excellent Talents in Fujian Province University under Grant No. JA13021.

Author information

Authors and Affiliations

  1. Fujian Provincial Key Lab of the Network Computing and Intelligent Information Processing, College of Mathematics and Computer Science, Fuzhou University, Fuzhou, Fujian, China

    Yuzhen Niu, Lingling Ke & Wenzhong Guo

  2. College of Mathematics and Computer Science, Fuzhou University, Qi Shan Campus, 2 Xue Yuan Road, University Town, Fuzhou, 350116, Fujian, People’s Republic of China

    Wenzhong Guo

Authors
  1. Yuzhen Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lingling Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenzhong Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenzhong Guo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Niu, Y., Ke, L. & Guo, W. Evaluation of visual saliency analysis algorithms in noisy images. Machine Vision and Applications 27, 915–927 (2016). https://doi.org/10.1007/s00138-016-0782-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-016-0782-6

Keywords

Search

Navigation