Abstract
We present a novel hybrid scheme for content-aware image retargeting that allows retargeting images into arbitrary dimensions while preserving visually prominent features and minimizing global information loss. One of the novelties in our scheme is an optimized importance map incorporating the impacts of the gradient map, context-aware saliency map, skin map and Canny edge map. Another novelty is a systematic utilization of both seam carving and scaling for a good balance between information loss and image stretching, where the number of seam operations along each dimension is adaptively determined by a non-linear optimization process. Furthermore, a switching factor is added to the optimization for interactive user control of the switching point between information loss and image stretching. In addition, we propose an optional step to accelerate seam carving by restricting the optimal seam search to a down-sampled thumbnail and the local regions of the input image.
Similar content being viewed by others
References
Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection, Frequency-tuned salient region detection Computer Vision and Pattern Recognition, 1597– 1604
Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. ACM Trans Graph (TOG) 26(3):10
Barnes C, Shechtman E, Dan BG, Finkelstein A (2010) The generalized patchmatch correspondence algorithm. Lect Notes Comput Sci 6313:29–43
Chen R, Freedman D, Karni Z, Gotsman C, Liu L (2010) Content-aware image resizing by quadratic programming, IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 1–8
Cheng M-M, Zhang G-X, Mitra NJ, Huang X, Hu S-M (2011) Global contrast based salient region detection, 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 409–416
Domingues D, Alahi A, Vandergheynst P (2010) Stream carving: an adaptive seam carving algorithm, 17th IEEE International Conference on Image Processing, 901–904
Dong W, Zhou N, Paul J-C, Zhang X (2009) Optimized image resizing using seam carving and scaling. ACM Trans Graph (TOG) 28(5):125
Dong W-M, Bao G-B, Zhang X-P, Paul J-C (2012) Fast multi-operator image resizing and evaluation. J Comput Sci Technol 27(1):121–134
Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139
Gal R, Sorkine O, Cohen-Or D (2006) Feature-aware texturing, Proceedings of the 17th Eurographics conference on Rendering Techniques, 297–303
Genesis Microchip (2002) gm6010/gm6015 Programming Guide, California US: Genesis Microchip Company, 85-90
Goferman S., Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915–1926
Guo Y, Liu F, Shi J, Zhou ZH, Gleicher M (2009) Image retargeting using mesh parametrization. IEEE Trans Multimedia 11(5):856–867
Hsu R-L, Abdel-Mottaleb M, Jain AK (2002) Face detection in color images. IEEE Trans Pattern Anal Mach Intell 24(5):696–706
Hwang D-S, Chien S-Y (2008) Content-aware image resizing using perceptual seam carving with human attention model, IEEE International Conference on Multimedia and Expo, 1029–1032
Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259
Jiang B, Zhang L, Lu H, Yang C, Yang M-H (2013) Saliency detection via absorbing markov chain, IEEE International Conference on Computer Vision, 1665–1672
Jonathan H, Christof K, Pietro P (2007) Graph-based visual saliency, Advances in Neural Information Processing Systems, 545–552
Liang Y, Su Z, Luo X (2012) Patchwise scaling method for content-aware image resizing. Signal Process 92(5):1243–1257
Liu F, Gleicher M (2005) Automatic image retargeting with fisheye-view war**, ACM UIST, 153– 162
Liu YJ, Luo X, Xuan YM, Chen WF, Fu XL (2011) Image retargeting quality assessment. Comput Graphics Forum 30(2):583–592
Ma L, Lin W, Deng C, Ngan KN (2012) Image retargeting quality assessment: a study of subjective scores and objective metrics. IEEE J Sel Top Sign Proces 6(6):626–639
Mansfield A, Gehler P, Gool LV, Rother C (2010) Scene carving: Scene consistent image retargeting, ECCV, 143–156
Mansfield A, Gehler P, Van Gool L, et al. (2012) Visibility maps for improving seam carving, Trends and Topics in Computer Vision, 131–144
Niu Y, Liu F, Li X, Gleicher M (2012) Image resizing via non-homogeneous war**. Multimedia Tools Appl 56(3):485–508
Rubinstein M, Shamir A, Avidan S (2008) Improved seam carving for video retargeting. ACM Transactions on Graphics (TOG) 27(3):16
Rubinstein M, Shamir A, Avidan S, Pierre (2009) Multi-operator media retargeting. ACM Trans Graph (TOG) 28(3):23
Rubinstein M, Gutierrez D, Sorkine O, Shamir A (2010) A comparative study of image retargeting. ACM Trans Graph (TOG) 29(6):160
Shi M, Yang L, Peng G, Xu D (2010) A content-aware image resizing method with prominent object size adjusted, Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology, 175–176
Simakov D, Caspi Y, Shechtman E, Irani M (2008) Summarizing visual data using bidirectional similarity, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1–8
Subramanian S, Kumar K, Mishra BP, et al. (2008) Fuzzy logic based content protection for image resizing by seam carving, IEEE Conference on Soft Computing in Industrial Applications, 78–83. SMCia’08.. IEEE, 2008:
Sun J, Haibin L (2011) Scale and object aware image retargeting for thumbnail browsing, IEEE International Conference on Computer Vision (ICCV), 1511–1518
Sun J, Haibin L (2013) Scale and object aware image thumbnailing. Int J Comput Vis 104(2):135– 153
Thilagam K, Karthikeyan S (2012) Optimized image resizing using piecewise seam carving. Int J Comput Appl 42(14):24–30
Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154
Wang Y-S, Tai C-L, Sorkine O, Lee T-Y (2008) Optimized scale-and-stretch for image resizing. ACM Trans Graph (TOG) 27(5):118
Wolf L, Guttmann M, Cohen-Or D (2007) Non-homogeneous content-driven video-retargeting, IEEE 11th International Conference on Computer Vision, 1–6
Wu L, Cao L, Wang J, Liu S (2013) Content aware metric for image resizing assessment, The Era of Interactive Media, 255–265
Yang J, Lu W, Waibel A (1997) Skin-color modeling and adaptation, 687–694
Zhang G-X, Cheng M-M, Hu S-M, Martin RR (2009) A Shape-Preserving approach to image resizing. Comput Graphics Forum 28(7):1897–1906
Acknowledgements
Authors thank three anonymous reviewers for their valuable suggestions to improve the presentation of this paper. This work is supported by the National Natural Science Foundation of China (Grants No.s 61303083, 61472223).
Author information
Authors and Affiliations
Corresponding authors
Rights and permissions
About this article
Cite this article
Zhang, Y., Sun, Z., Jiang, P. et al. Hybrid image retargeting using optimized seam carving and scaling. Multimed Tools Appl 76, 8067–8085 (2017). https://doi.org/10.1007/s11042-016-3318-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-016-3318-1