Abstract
In this chapter, we review the existing works regarding visual quality assessment in computer graphics. This broad area of research includes many sub-domains which make intensive use of quality assessment and/or artifact visibility evaluation: geometry processing, rendering, HDR imaging, tone-map**, and stereo vision. For each of these sub-domains, we present the existing objective quality metrics, the subjective quality experiments as well as an evaluation and comparison of their performance. We broadly classify these existing works into image-based (i.e., evaluating artifacts introduced in 2D rendered images and videos) and model-based approaches (i.e., artifacts introduced on the 3D models themselves). Finally, the last part presents the emerging trends and main future directions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Akyüz, A.O., Fleming, R., Riecke, B.E., Reinhard, E., Bulthoff, H.H.: Do HDR displays support LDR content? A psychophysical evaluation. ACM Transactions on Graphics 26(3), article no. 38 (2007)
Akyüz, A.O., Reinhard, E.: Perceptual evaluation of tone reproduction operators using the Cornsweet-Craik-O’Brien illusion. ACM Transactions on Applied Perception 4(4), 1–29 (2008)
Allan, R., Terry, M.E.: Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons. Biometrica 39(3/4), 324–345 (1952)
Ashikhmin, M., Goyal, J.: A reality check for tone map** operators. ACM Transactions on Applied Perception 3(4), 399–411 (2006)
Aydın, T.O., Mantiuk, R., Myszkowski, K., Seidel, H.P.: Dynamic range independent image quality assessment. ACM Transactions on Graphics (Proc. of SIGGRAPH) 27(3), 69 (2008)
Aydın, T.O., Mantiuk, R., Seidel, H.P.: Extending quality metrics to full luminance range images. In: Proceedings of SPIE, pp. 68,060B–10. Spie (2008). DOI 10.1117/12.765095
Aydın, T.O., Čadík, M., Myszkowski, K., Seidel, H.P.: Video quality assessment for computer graphics applications. ACM Transactions on Graphics 29(6), 1 (2010). DOI 10.1145/1882261.1866187
Banterle, F., Ledda, P., Debattista, K., Bloj, M., Artusi, A., Chalmers, A.: A Psychophysical Evaluation of Inverse Tone Map** Techniques. Computer Graphics Forum 28(1), 13–25 (2009). DOI 10.1111/j.1467-8659.2008.01176.x
Blackwell, H.: Contrast thresholds of the human eye. Journal of the Optical Society of America 36(11), 624–632 (1946)
Boitard, R., Cozot, R., Thoreau, D., Bouatouch, K.: Temporal coherency in video tone map**, a survey. In: HDRi2013 - First International Conference and SME Workshop on HDR imaging, Xx, p. no. 1 (2013)
Bolin, M.R., Meyer, G.W.: A frequency based ray tracer. In: Proc. of SIGGRAPH ’95, pp. 409–418 (1995)
Bolin, M.R., Meyer, G.W.: A perceptually based adaptive sampling algorithm. In: Proceedings of the 25th annual conference on Computer graphics and interactive techniques - SIGGRAPH ’98, pp. 299–309. ACM Press, New York, New York, USA (1998). DOI 10.1145/280814.280924
Bulbul, A., Capin, T., Lavoue, G., Preda, M.: Measuring Visual Quality of 3D Polygonal Models. IEEE Signal Processing Magazine 28(6), 80–90 (2011)
Cater, K., Chalmers, A., Ward, G.: Detail to Attention: Exploiting Visual Tasks for Selective Rendering. Proc. of Eurographics workshop on Rendering pp. 270–280 (2003)
Cho, J., Prost, R., Jung, H.: An oblivious watermarking for 3-D polygonal meshes using distribution of vertex norms. IEEE Transactions on Signal Processing 55(1), 142–155 (2007)
Cignoni, P., Rocchini, C., Scopigno, R.: Metro: Measuring Error on Simplified Surfaces. Computer Graphics Forum 17(2), 167–174 (1998). DOI 10.1111/1467-8659.00236
Cleju, I., Saupe, D.: Evaluation of supra-threshold perceptual metrics for 3D models. In: Symposium on Applied Perception in Graphics and Visualization. ACM Press (2006). DOI 10.1145/1140491.1140499
Corsini, M., Gelasca, E.D., Ebrahimi, T., Barni, M.: Watermarked 3-D Mesh Quality Assessment. IEEE Transactions on Multimedia 9(2), 247–256 (2007)
Corsini, M., Larabi, M.C., Lavoué, G., Petík, O., Váša, L., Wang, K.: Perceptual Metrics for Static and Dynamic Triangle Meshes. Computer Graphics Forum 32(1), 101–125 (2013). DOI 10.1111/cgf.12001
Daly, S.: The Visible Differences Predictor: An Algorithm for the Assessment of Image Fidelity. In: A.B. Watson (ed.) Digital Images and Human Vision, pp. 179–206. MIT Press (1993)
Delahunt, P.B., Zhang, X., Brainard, D.H.: Perceptual image quality: Effects of tone characteristics. Journal of Electronic Imaging 14(2), 1–12 (2005). DOI 10.1117/1.1900134
Drago, F., L. Martens, W., Myszkowski, K., Sidel, H.P.: Perceptual Evaluation of Tone Map** Operators with Regard to Similarity and Preference. Tech. rep., MPI Informatik (2002)
Dumont, R., Pellacini, F., Ferwerda, J.A.: Perceptually-driven decision theory for interactive realistic rendering. ACM Transactions on Graphics 22(2), 152–181 (2003). DOI 10.1145/636886.636888
Eilertsen, G., Wanat, R., Mantiuk, R.K., Unger, J.: Evaluation of Tone Map** Operators for HDR-Video. Computer Graphics Forum (Proc. of Pacific Graphics) 32(7), 275–284 (2013)
Engeldrum, P.: Psychometric scaling: a toolkit for imaging systems development. Imcotek Press (2000)
Ferwerda, J.A., Shirley, P., Pattanaik, S.N., Greenberg, D.P.: A model of visual masking for computer graphics. In: Proc. of SIGGRAPH ’97, pp. 143–152. ACM Press, New York, New York, USA (1997). DOI 10.1145/258734.258818
Georgeson, M.A., Sullivan, G.D.: Contrast constancy: deblurring in human vision by spatial frequency channels. J. Physiol. 252(3), 627–656 (1975)
Guthe, M., Müller, G., Schneider, M., Klein, R.: BTF-CIELab: A Perceptual Difference Measure for Quality Assessment and Compression of BTFs. Computer Graphics Forum 28(1), 101–113 (2009). DOI 10.1111/j.1467-8659.2008.01299.x
Hadziabdic, K.K., Telalovic, J.H., Mantiuk, R.: Comparison of deghosting algorithms for multi-exposure high dynamic range imaging. In: Proc. of Spring Conference on Computer Graphics, pp. 1–8 (2013)
Herzog, R., Čadík, M., Aydın, T.O., Kim, K.I., Myszkowski, K., Seidel, H.P.: NoRM: No-Reference Image Quality Metric for Realistic Image Synthesis. Computer Graphics Forum 31(2pt3), 545–554 (2012). DOI 10.1111/j.1467-8659.2012.03055.x
Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259 (1998). DOI 10.1109/34.730558
Karni, Z., Gotsman, C.: Spectral compression of mesh geometry. In: ACM Siggraph, pp. 279–286 (2000)
Karni, Z., Gotsman, C.: Compression of Soft-Body animation sequences. Computers & Graphics 28(1), 25–34 (2004)
Kelly, D.H.: Motion and vision. {II}. Stabilized spatiotemporal threshold surface. Journal of Optical Society of America 69(10), 1340–1349 (1979)
Kim, K.J., Mantiuk, R., Lee, K.H.: Measurements of achromatic and chromatic contrast sensitivity functions for an extended range of adaptation luminance. In: B.E. Rogowitz, T.N. Pappas, H. de Ridder (eds.) Human Vision and Electronic Imaging, p. 86511A (2013). DOI 10.1117/12.2002178
Korshunov, P., Ebrahimi, T.: Influence of Context and Content on Tone-map** Operators. In: HDRi2013 - First International Conference and SME Workshop on HDR imaging, p. no. 2 (2013)
Krawczyk, G., Myszkowski, K., Seidel, H.P.: Contrast Restoration by Adaptive Countershading. Computer Graphics Forum 26(3), 581–590 (2007). DOI 10.1111/j.1467-8659.2007.01081.x
Kuang, J., Heckaman, R., Fairchild, M.D.: Evaluation of HDR tone-map** algorithms using a high-dynamic-range display to emulate real scenes. Journal of the Society for Information Display 18(7), 461–468 (2010). DOI 10.1889/JSID18.7.461
Kuang, J., Yamaguchi, H., Johnson, G.M., Fairchild, M.D.: Testing HDR image rendering algorithms. In: Proc. IS&T/SID 12th Color Imaging Conference, pp. 315–320. Scotsdale, Arizona (2004)
Larkin, M., O’Sullivan, C.: Perception of Simplification Artifacts for Animated Characters. In: symposium on Applied perception in graphics and visualization, pp. 93–100 (2011)
Lavoué, G.: A local roughness measure for 3D meshes and its application to visual masking. ACM Transactions on Applied Perception (TAP) 5(4) (2009)
Lavoué, G.: A Multiscale Metric for 3D Mesh Visual Quality Assessment. Computer Graphics Forum 30(5), 1427–1437 (2011)
Lavoué, G., Cheng, I., Basu, A.: Perceptual Quality Metrics for 3D Meshes: Towards an Optimal Multi-Attribute Computational Model. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC) (2013)
Lavoué, G., Corsini, M.: A comparison of perceptually-based metrics for objective evaluation of geometry processing. IEEE Transactions on Multimedia 12(7), 636–649 (2010)
Lavoue, G., Drelie Gelasca, E., Dupont, F., Baskurt, A., Ebrahimi, T.: Perceptually driven 3D distance metrics with application to watermarking. In: SPIE, vol. 6312, pp. 63,120L–63,120L–12. SPIE (2006)
Ledda, P., Chalmers, A., Troscianko, T., Seetzen, H.: Evaluation of tone map** operators using a high dynamic range display. ACM Transactions on Graphics 24(3), 640–648 (2005)
Lindstrom, P.: Model Simplification using Image and Geometry-Based Metrics. Ph.D. thesis, Georgia Institute of Technology (2000)
Lindstrom, P., Turk, G.: Evaluation of memoryless simplification. IEEE Transactions on Visualization and Computer Graphics 5(2), 98–115 (1999). DOI 10.1109/2945.773803
Lindstrom, P., Turk, G.: Image Driven Simplification. ACM Transactions on Graphics 19(3), 204–241 (2000)
Liu, Y., Wang, J., Cho, S., Finkelstein, A., Rusinkiewicz, S.: A no-reference metric for evaluating the quality of motion deblurring. ACM Transactions on Graphics 32(6), 1–12 (2013). DOI 10.1145/2508363.2508391
Lubin, J.: A visual discrimination model for imaging system design and evaluation. In: E. Peli (ed.) Vision Models for Target Detection and Recognition, pp. 245–283. World Scientific Publishing Company (1995)
Luebke, D., Hallen, B.: Perceptually driven simplification for interactive rendering. In: Rendering Techniques 2001: Proceedings of the Eurographics Workshop, p. 223 (2001)
Luebke, D., Hallen, B., Newfield, D., Watson, B.: Perceptually Driven Simplification Using Gaze-Directed Rendering. In: EGSR, pp. 223–234 (2001)
Mantiuk, R., Daly, S., Myszkowski, K., Seidel, H.: Predicting visible differences in high dynamic range images: model and its calibration. In: Human Vision and Electronic Imaging, pp. 204–214 (2005)
Mantiuk, R., Kim, K.J., Rempel, A.G., Heidrich, W.: HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph (Proc. SIGGRAPH) 30(4), 1 (2011). DOI 10.1145/2010324.1964935
Mantiuk, R.K., Tomaszewska, A., Mantiuk, R.: Comparison of four subjective methods for image quality assessment. Computer Graphics Forum 31(8), 2478–2491 (2012)
Masia, B., Agustin, S., Fleming, R.W., Sorkine, O., Gutierrez, D.: Evaluation of reverse tone map** through varying exposure conditions. ACM Transactions on Graphics 28(5), 1 (2009). DOI 10.1145/1618452.1618506
McCann, J., Rizzi, A.: Veiling glare: the dynamic range limit of hdr images. In: Proc. of HVEI XII, vol. 6492, pp. 649,213–649,213. International Society for Optics and Photonics (2007)
McCann, J.J., Rizzi, A.: The Art and Science of HDR Imaging (Google eBook). John Wiley & Sons (2011)
Menzel, N., Guthe, M.: Towards Perceptual Simplification of Models with Arbitrary Materials. Computer Graphics Forum 29(7), 2261–2270 (2010). DOI 10.1111/j.1467-8659.2010.01815.x
Mullen, K.T.: The contrast sensitivity of human colour vision to red-green and blue-yellow chromaic gratings. Journal of Physiolohy 359, 381–400 (1985)
Myszkowski, K.: The visible differences predictor: Applications to global illumination problems. In: Rendering techniques’ 98: proceedings of the Eurographics Workshop in Vienna, Austria, June 29-July 1, 1998, p. 223. Springer Verlag Wien (1998)
Myszkowski, K., Rokita, P., Tawara, T.: Perceptually-informed accelerated rendering of high quality walkthrough sequences. In: Eurographics Workshop on Rendering, vol. 99, pp. 5–18 (1999)
Myszkowski, K., Tawara, T., Akamine, H., Seidel, H.P.: Perception-guided global illumination solution for animation rendering. In: Proc. of SIGGRAPH’01, pp. 221–230. ACM, New York, NY, USA (2001). DOI 10.1145/383259.383284
O’Donovan, P., Agarwala, A., Hertzmann, A.: Color compatibility from large datasets. ACM Transactions on Graphics 30(4), 1 (2011). DOI 10.1145/2010324.1964958
O’Sullivan, C., Howlett, S., Morvan, Y.: Perceptually adaptive graphics. Eurographics State of the Art Reports pp. 141–164 (2004)
Pan, Y., Cheng, I., Basu, A.: Quality metric for approximating subjective evaluation of 3-D objects. IEEE Transactions on Multimedia 7(2), 269–279 (2005)
Petit, J., Mantiuk, R.K.: Assessment of video tone-map**: Are cameras S-shaped tone-curves good enough? Journal of Visual Communication and Image Representation 24, 1020–1030 (2013). DOI 10.1016/j.jvcir.2013.06.014
Ponomarenko, N., Lukin, V., Zelensky, A., Egiazarian, K., Carli, M., Battisti, F.: TID2008 - A database for evaluation of full-reference visual quality assessment metrics. Advances of Modern Radioelectronics 10, 30–45 (2009)
Qu, L., Meyer, G.: Perceptually guided polygon reduction. IEEE Transactions on Visualization and Computer Graphics 14(5), 1015–1029 (2008). DOI 10.1109/TVCG.2008.51
Ramanarayanan, G., Ferwerda, J., Walter, B.: Visual equivalence: towards a new standard for image fidelity. ACM Transactions on Graphics (TOG) 26(3), 76 (2007). DOI 10.1145/1276377.1276472
Ramasubramanian, M., Pattanaik, S.N., Greenberg, D.P.: A perceptually based physical error metric for realistic image synthesis. In: Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH ’99, pp. 73–82. ACM Press, New York, New York, USA (1999). DOI 10.1145/311535.311543
Reddy, M.: SCROOGE: Perceptually-Driven Polygon Reduction. Computer Graphics Forum 15(4), 191–203 (1996)
Rogowitz, B.E., Holly E. Rushmeier: Are image quality metrics adequate to evaluate the quality of geometric objects? Proceedings of SPIE pp. 340–348 (2001)
Rubinstein, M., Gutierrez, D., Sorkine, O., Shamir, A.: A comparative study of image retargeting. ACM Transactions on Graphics 29(6), 1 (2010). DOI 10.1145/1882261.1866186
Rushmeier, H., Rogowitz, B., Piatko, C.: Perceptual issues in substituting texture for geometry. In: SPIE, pp. 372–383. International Society for Optical Engineering; 1999 (2000)
Schroeder, W., Zarge, J., Lorensen, W.: Decimation of triangle meshes. In: ACM Siggraph, pp. 65–70 (1992)
Secord, A., Lu, J., Finkelstein, A., Singh, M., Nealen, A.: Perceptual models of viewpoint preference. ACM Transactions on Graphics 30(5), 1–12 (2011). DOI 10.1145/2019627.2019628
Silva, S., Santos, B., Ferreira, C.: Comparison of methods for the simplification of mesh models using quality indices and an observer study. SPIE pp. 64,921L–64,921L–12 (2007)
Silva, S., Santos, B.S., Ferreira, C., Madeira, J.: A Perceptual Data Repository for Polygonal Meshes. 2009 Second International Conference in Visualisation pp. 207–212 (2009)
Silverstein, D., Farrell, J.: Efficient method for paired comparison. Journal of Electronic Imaging 10, 394 (2001). DOI 10.1117/1.1344187
Smith, K., Krawczyk, G., Myszkowski, K.: Beyond tone map**: Enhanced depiction of tone mapped HDR images. Computer Graphics Forum 25(3), 427–438 (2006)
Sorkine, O., Cohen-Or, D., Toldeo, S.: High-pass quantization for mesh encoding. In: Eurographics Symposium on Geometry Processing, pp. 42–51 (2003)
Tian, D., AlRegib, G.: FQM: A Fast Quality Measure for Efficient Transmission of Textured 3D Models. In: ACM Multimedia, pp. 684–691 (2004)
Torkhani, F., Wang, K., Chassery, J.m.: A Curvature Tensor Distance for Mesh Visual Quality Assessment. In: International Conference on Computer Vision and Graphics (2012)
Trentacoste, M., Mantiuk, R., Heidrich, W., Dufrot, F.: Unsharp Masking, Countershading and Halos: Enhancements or Artifacts? Computer Graphics Forum 31(2pt3), 555–564 (2012). DOI 10.1111/j.1467-8659.2012.03056.x
Vasa, L., Skala, V.: A Perception Correlated Comparison Method for Dynamic Meshes. IEEE Trans. on Visualization and Computer Graphics 17(2), 220–230 (2011)
Váša, L., Rus, J.: Dihedral Angle Mesh Error: a fast perception correlated distortion measure for fixed connectivity triangle meshes. Computer Graphics Forum 31(5) (2012)
Čadík, M., Herzog, R., Mantiuk, R., Mantiuk, R., Myszkowski, K., Seidel, H.P.: Learning to Predict Localized Distortions in Rendered Images. Computer Graphics Forum (Proc. of Pacific Graphics) 32(7), 401–410 (2013)
Čadík, M., Herzog, R., Mantiuk, R.K., Myszkowski, K., Seidel, H.P., Čadík, M.: New Measurements Reveal Weaknesses of Image Quality Metrics in Evaluating Graphics Artifacts. ACM Trans. Graph (Proc. SIGGRAPH Asia) 31(6), 147 (2012). DOI 10.1145/2366145.2366166
Čadík, M., Wimmer, M., Neumann, L., Artusi, A.: Evaluation of HDR tone map** methods using essential perceptual attributes. Computers & Graphics 32(3), 330–349 (2008). DOI 10.1016/j.cag.2008.04.003
Villa, C., Labayrade, R.: Psychovisual assessment of tone-map** operators for global appearance and colour reproduction. In: Proc. of Colour in Graphics Imaging and Vision 2010, pp. 189–196. Joensuu, Finland (2010)
VQEG: Final report from the video quality experts group on the validation of objective models of video quality assessment. Tech. rep., Video Quality Experts Group (2000)
Walter, B., Pattanaik, S.N., Greenberg, D.P.: Using Perceptual Texture Masking for Efficient Image Synthesis. Computer Graphics Forum 21(3), 393–399 (2002). DOI 10.1111/1467-8659.t01-1-00599
Wang, K., Lavoué, G., Denis, F., Baskurt, A.: Robust and blind mesh watermarking based on volume moments. Computers & Graphics 35(1), 1–19 (2011)
Wang, K., Torkhani, F., Montanvert, A.: A Fast Roughness-Based Approach to the Assessment of 3D Mesh Visual Quality. Computers & Graphics (2012)
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13(4), 600–612 (2004)
Watson, A., Ahumada Jr, A.: A standard model for foveal detection of spatial contrast. Journal of Vision 5(9), 717–740 (2005)
Watson, A.B.: The cortex transform: Rapid computation of simulated neural images. Computer Vision, Graphics, and Image Processing 39(3), 311–327 (1987). DOI 10.1016/S0734-189X(87)80184-6
Watson, B., Friedman, A., McGaffey, A.: Measuring and predicting visual fidelity. ACM Siggraph pp. 213–220 (2001)
Williams, N., Luebke, D., Cohen, J., Kelley, M., Schubert, B.: Perceptually Guided Simplification of Lit, Textured Meshes. In: ACM Symposium on Interactive 3D Graphics, pp. 113–121 (2003)
Wilson, H.R.: A transducer function for threshold and suprathreshold human vision. Biological Cybernetics 38(3), 171–178 (1980). DOI 10.1007/BF00337406
Yee, H.: Perceptual Metric for Production Testing. Journal of Graphics Tools 9(4), pages 33–40 (2004)
Yee, H., Pattanaik, S., Greenberg, D.P.: Spatiotemporal sensitivity and visual attention for efficient rendering of dynamic environments. ACM Transactions on Graphics 20(1), 39–65 (2001). DOI 10.1145/383745.383748
Yeganeh, H., Wang, Z.: Objective quality assessment of tone-mapped images. IEEE Transactions on Image Processing 22(2), 657–67 (2013). DOI 10.1109/TIP.2012.2221725
Yoshida, A., Blanz, V., Myszkowski, K., Seidel, H.P.: Perceptual evaluation of tone map** operators with real world scenes. In: Proc. of SPIE Human Vision and Electronic Imaging X, vol. 5666, pp. 192–203. San Jose, CA (2005)
Yoshida, A., Mantiuk, R., Myszkowski, K., Seidel, H.P.: Analysis of reproducing real-world appearance on displays of varying dynamic range. Computer Graphics Forum 25(3), 415–426 (2006)
Zhang, X., Wandell, B.A.: A spatial extension of CIELAB for digital color-image reproduction. Journal of the Society for Information Display 5(1), 61 (1997). DOI 10.1889/1.1985127
Zhu, Q., Zhao, J., Du, Z., Zhang, Y.: Quantitative analysis of discrete 3D geometrical detail levels based on perceptual metric. Computers & Graphics 34(1), 55–65 (2010). DOI 10.1016/j.cag.2009.10.004
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Lavoué, G., Mantiuk, R. (2015). Quality Assessment in Computer Graphics. In: Deng, C., Ma, L., Lin, W., Ngan, K. (eds) Visual Signal Quality Assessment. Springer, Cham. https://doi.org/10.1007/978-3-319-10368-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-10368-6_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10367-9
Online ISBN: 978-3-319-10368-6
eBook Packages: EngineeringEngineering (R0)