SpringerLink
Log in

Pyramidal modeling of geometric distortions for retargeted image quality evaluation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Content-aware retargeting methods are used to adjust images to different resolutions and aspect ratios with low deformation and information loss in salient regions. Effective objective quality assessment of retargeted images can provide a way to improve retargeting methods. The non-uniform geometrical degradations caused by retargeting algorithms make it impossible to use traditional image quality assessment metrics for retargeted images. Although some quality evaluation methods have been proposed till now, the resulted quality scores are not well correlated with the subjective ones. In this paper we propose a pyramidal global-to-local pooling method to combine pixel/block deformation measures. In each level of locality, the Euclidean distance between the retargeted and original image is used as an individual feature. Therefore, in addition to the global summation, assessment of local deformations, contributes toward better quality evaluation. Learning a regression model based on the extracted features results in better performance compared to relevant existing retargeted image quality methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. ACM Trans Graph 26(3):10

    Article  Google Scholar 

  2. Dong W, Zhou N, Paul J-C, Zhang X (2009) Optimized image resizing using seam carving and scaling. ACM Trans Graph 28(5):125

    Article  Google Scholar 

  3. Fang Y, Chen Z, Lin W, Lin C-W (2012) Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans Image Process 21(9):3888–3901

    Article  MathSciNet  MATH  Google Scholar 

  4. Fang Y, Zeng K, Wang Z, Lin W, Fang Z, Lin C-W (2014) Objective quality assessment for image retargeting based on structural similarity. IEEE J Emerging Sel Top Circuits Syst 4(1):95–105

    Article  Google Scholar 

  5. Guo Y, Liu F, Shi J, Zhou ZH, Gleicher M (2009) Image retargeting using mesh parametrization. IEEE Trans Multimedia 11(5):856–867

    Article  Google Scholar 

  6. Han J, Farin D, de With P (2011) A mixed-reality system for broadcasting sports video to mobile devices. IEEE MultiMedia 18(2):72–84

    Article  Google Scholar 

  7. Hsu C-C, Lin C-W, Fang Y, Lin W (2014) Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss. IEEE J Sel Top Sign Proces 8(3):377–389

    Article  Google Scholar 

  8. Karimi M, Samavi S, Karimi N, Soroushmehr SR, Lin W, Najarian K (2017) Quality assessment of retargeted images by Salient Region Deformity Analysis. J Vis Commun Image Represent 43:108–118

    Article  Google Scholar 

  9. Karni Z, Freedman D, Gotsman C (2009) Energy-based image deformation. Comput Graphics Forum 28(5):1257–1268

    Article  Google Scholar 

  10. Kopf S, Haenselmann T, Kiess J, Guthier B, Effelsberg W (2011) Algorithms for video retargeting. Multimed Tools Appl 51(2):819–886

    Article  Google Scholar 

  11. Kopf S, Guthier B, Farin D, and Han J (2011) Analysis and retargeting of ball sports video. In: IEEE Workshop on Applications of Computer Vision (WACV 2011), pp. 9–14

  12. Krähenbühl P, Lang M, Hornung A, Gross M (2009) A system for retargeting of streaming video. ACM Trans Graph 28(5):126

    Article  Google Scholar 

  13. Liang Y, Liu Y-J, and Gutierrez D (2016) Objective quality prediction of image retargeting algorithms. IEEE Trans Vis Comput Graph (99):1

  14. Lin J, Zhu L, Chen Z, and Chen X (2015) Objective quality assessment for image retargeting based on hybrid distortion pooled model. In 7th International Workshop on Quality of Multimedia Experience (QoMEX 2015), pp. 1–6

  15. Liu C, Yuen J, Torralba A, Sivic J, and Freeman WT (2008) Sift flow: dense correspondence across different scenes. In: 10th European Conference on Computer Vision (ECCV 2008), ed: Springer, pp. 28–42

  16. Liu YJ, Luo X, Xuan YM, Chen WF, Fu XL (2011) Image retargeting quality assessment. in Computer Graphics Forum 30(2):583–592

    Article  Google Scholar 

  17. Liu A, Lin W, Chen H, Zhang P (2015) Image retargeting quality assessment based on support vector regression. Signal Process Image Commun 39(2):444–456

    Article  Google Scholar 

  18. Ma L, Lin W, Deng C, Ngan KN (2012) Image retargeting quality assessment: a study of subjective scores and objective metrics. IEEE J Sel Top Sign Proces 6(6):626–639

    Article  Google Scholar 

  19. Ma L, Xu L, Zeng H, Ngan KN, and Deng C (2014) How does the shape descriptor measure the perceptual quality of the retargeting image? In: IEEE International Conference on Multimedia and Expo Workshops (ICMEW 2014), 2014, pp. 1–6

  20. Manjunath BS, Ohm J-R, Vasudevan VV, Yamada A (2001) Color and texture descriptors. IEEE Trans Circuits Syst Video Technol 11(6):703–715

    Article  Google Scholar 

  21. Mantiuk R, Kim KJ, Rempel AG, Heidrich W (2011) HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans Graph 30(4):40

    Article  Google Scholar 

  22. Pele O and Werman M (2009) Fast and robust earth mover's distances. In: IEEE 12th international conference on Computer vision (ICCV 2009), pp. 460–467

  23. Pritch Y, Kav-Venaki E, Peleg S (2009) Shift-map image editing. In: IEEE 12th International Conference on Computer Vision (ICCV 2009), pp. 151–158

  24. Qiu Z, Ren T, Liu Y, Bei J, and Yang Y (2013) Multi-operator image retargeting based on automatic quality assessment. In: Seventh International Conference on Image and Graphics (ICIG 2013) pp. 428–433

  25. Ren T, and Wu G (2010) Automatic image retargeting evaluation based on user perception. In: 17th IEEE International Conference on Image Processing (ICIP 2010), pp. 1569–1572

  26. Ren T, Guo Y, Wu G, Zhang F (2008) Constrained sampling for image retargeting. In: IEEE International Conference on Multimedia and Expo (ICME 2008), pp. 1397–1400

  27. Rubinstein M, Shamir A, Avidan S (2009) Multi-operator media retargeting," in ACM Transactions on Graphics (TOG), p. 23

  28. Rubinstein M, Gutierrez D, Sorkine O, and Shamir A (2012) Retarget me a benchmark for image retargeting, ed, Online: http://people.csail.mit.edu/mrub/retargetme/, available 2016

  29. Rubner Y, Tomasi C, Guibas LJ (2000) The earth mover's distance as a metric for image retrieval. Int J Comput Vis 40(2):99–121

    Article  MATH  Google Scholar 

  30. Rychetsky M (2001) Algorithms and architectures for machine learning based on regularized neural networks and support vector approaches, Shaker

  31. Schölkopf B and Smola AJ (2002) Learning with kernels: support vector machines, regularization, optimization, and beyond: MIT press

  32. Simakov D, Caspi Y, Shechtman E, and Irani M (2008) Summarizing visual data using bidirectional similarity. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2008), pp. 1–8

  33. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  34. Wang Y-S, Tai C-L, Sorkine O, Lee T-Y (2008) Optimized scale-and-stretch for image resizing. ACM Trans Graph 27(5):118

    Article  Google Scholar 

  35. Wang J, Bei J, Ren T, Liu Y, and Xu B (2012) Image retargeting assessment based on salient region similarity In: 4th International Conference on Internet Multimedia Computing and Service, pp. 140–143

  36. Wolf L., Guttmann M, Cohen-Or D (2007) Non-homogeneous content-driven video-retargeting. In: IEEE 11th International Conference on Computer Vision ( ICCV 2007), pp. 1–6

  37. Zhang J and Kuo C-CJ (2014) An objective Quality of Experience (QoE) assessment index for retargeted images. In The ACM International Conference on Multimedia, pp. 257–266

  38. Zhang Y, Fang Y, Lin W, Zhang X, Li L (2016) Backward registration rased Aspect Ratio Similarity (ARS) for image retargeting quality assessment. IEEE Trans Image Process 25(9):4286–4297

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Isfahan University of Technology, Isfahan, 84156-83111, Iran

    Maryam Karimi, Shadrokh Samavi & Nader Karimi

  2. Michigan Center for Integrative Research in Critical Care, University of Michigan, Ann Arbor, 48109, USA

    Shadrokh Samavi, S. M. Reza Soroushmehr & Kayvan Najarian

  3. Department of Emergency Medicine, University of Michigan, Ann Arbor, 48109, USA

    S. M. Reza Soroushmehr & Kayvan Najarian

  4. School of Computer Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Weisi Lin

  5. Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, 48109, USA

    Kayvan Najarian

Authors
  1. Maryam Karimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shadrokh Samavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nader Karimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. S. M. Reza Soroushmehr
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Weisi Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kayvan Najarian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. M. Reza Soroushmehr.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karimi, M., Samavi, S., Karimi, N. et al. Pyramidal modeling of geometric distortions for retargeted image quality evaluation. Multimed Tools Appl 77, 13799–13820 (2018). https://doi.org/10.1007/s11042-017-4994-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4994-1

Keywords

Search

Navigation