SpringerLink
Log in

Evaluating User Experience in Information Visualization Systems: UXIV an Evaluation Questionnaire

  • Conference paper
  • First Online:
Human Interface and the Management of Information (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14015))

Included in the following conference series:

  • 919 Accesses

Abstract

Understanding how people interact with data and why users prefer one system over another is necessary to develop successful solutions for information visualization. Evaluating the users’ experience with applications based on interaction with data is challenging due to the several parameters involved. It requires evaluating the experience offered by the system, affected data, and the analysis process supported. Existing measurement tools do not take all of these aspects into account. This study investigated the gaps in an existing user experience evaluation tool regarding the interaction with data. A thematic analysis identified the missing themes and grounded our definition of evaluation dimensions. A combination of literature review and designer collaboration resulted in the evaluation statements. Our contribution consists of a questionnaire to support evaluating specific aspects of experience and relevant dimensions for measuring the interaction with data in information visualization systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://en.wikipedia.org/wiki/Questionnaire.

  2. 2.

    The ethical committee of the University of Campinas approved this study (#18927119.9.0000.5404).

  3. 3.

    Available in October 2019 in https://www.kenflerlage.com/2016/08/whats-happiest-country-in-world.html.

  4. 4.

    The HDI module of the UXIV questionnaire is available online at https://forms.gle/3ioXom95ovzng5xn7.

  5. 5.

    The other modules used by UXIV are original from meCUE and are available online at http://mecue.de/english/download-en.html.

References

  1. Adagha, O., Levy, R.M., Carpendale, S.: Towards a product design assessment of visual analytics in decision support applications: a systematic review. J. Intell. Manuf. 28(7), 1623–1633 (2017)

    Article  Google Scholar 

  2. Attfield, S., Kazai, G., Lalmas, M.: Towards a science of user engagement (position paper). In: WSDM Workshop on User Modelling for Web Applications (2011). http://www.dcs.gla.ac.uk/~mounia/Papers/engagement.pdf

  3. Bangor, A., Kortum, P., Miller, J.: Determining what individual SUS scores mean: adding an adjective rating scale. J. Usability Stud. 4(3), 114–123 (2009)

    Google Scholar 

  4. Borkin, M.A., et al.: Beyond memorability: visualization recognition and recall. IEEE Trans. Visual Comput. Graphics 22(1), 519–528 (2016). https://doi.org/10.1109/TVCG.2015.2467732

    Article  Google Scholar 

  5. Boy, J., Detienne, F., Fekete, J.D.: Storytelling in information visualizations: does it engage users to explore data? In: Conference on Human Factors in Computing Systems - Proceedings, pp. 1449–1458 (2015). https://doi.org/10.1145/2702123.2702452

  6. Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment semantic differential manikin and the. J. Behav. Therapy Exp. Psychiatry 25(I), 49–59 (1994). https://doi.org/10.1016/0005-7916(94)90063-9

  7. Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3(2), 77–101 (2006). https://doi.org/10.1191/1478088706qp063oa

    Article  Google Scholar 

  8. Carpendale, S.: Evaluating information visualizations. In: Kerren, A., Stasko, J.T., Fekete, J.-D., North, C. (eds.) Information Visualization. LNCS, vol. 4950, pp. 19–45. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-70956-5_2

    Chapter  Google Scholar 

  9. Darin, T., Coelho, B., Borges, B.: Which instrument should i use? Supporting decision-making about the evaluation of user experience. In: Marcus, A., Wang, W. (eds.) HCII 2019. LNCS, vol. 11586, pp. 49–67. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23535-2_4

    Chapter  Google Scholar 

  10. Davis, W.A.: A causal theory of enjoyment. Mind 91(362), 240–256 (1982)

    Article  Google Scholar 

  11. DeVellis, R.: Scale Development: Theory and Applications. Sage Publications, Thousand Oaks (2017)

    Google Scholar 

  12. Elmqvist, N.: Embodied human-data interaction. In: ACM CHI 2011 Workshop Embodied Interaction: Theory and Practice in HCI, vol. 1, pp. 104–107 (2011)

    Google Scholar 

  13. Gross, A., Bongartz, S.: Why do i like it? Investigating the product-specificity of user experience. In: NordiCHI 2012: Making Sense Through Design - Proceedings of the 7th Nordic Conference on Human-Computer Interaction, pp. 322–330 (2012). https://doi.org/10.1145/2399016.2399067

  14. Hassenzahl, M.: The thing and i: understanding the relationship between user and product. In: Blythe, M., Monk, A. (eds.) Funology 2. HIS, pp. 301–313. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-68213-6_19

    Chapter  Google Scholar 

  15. Hassenzahl, M., Tractinsky, N.: User experience - a research agenda. Behav. Inf. Technol. 25(2), 91–97 (2006). https://doi.org/10.1080/01449290500330331

    Article  Google Scholar 

  16. He, T., Isenberg, P., Dachselt, R., Isenberg, T.: Beauvis: a validated scale for measuring the aesthetic pleasure of visual representations. IEEE Trans. Visual Comput. Graphics 29(1), 363–373 (2022)

    Google Scholar 

  17. Hornung, H., Pereira, R., Baranauskas, M.C.C., Liu, K.: Challenges for human-data interaction – a semiotic perspective. In: Kurosu, M. (ed.) HCI 2015. LNCS, vol. 9169, pp. 37–48. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20901-2_4

    Chapter  Google Scholar 

  18. Isenberg, P., Zuk, T., Collins, C., Carpendale, S.: Grounded evaluation of information visualizations. In: Proceedings of the 2008 Workshop on BEyond Time and Errors: Novel Evaluation Methods for Information Visualization, pp. 1–8 (2008)

    Google Scholar 

  19. ISO 9241-11:2018 ergonomics of human-system interaction–part 11: usability: definitions and concepts. International Organization for Standardization (2018). https://www.iso.org/obp/ui/

  20. Lam, H., Bertini, E., Isenberg, P., Plaisant, C., Carpendale, S.: Empirical studies in information visualization: seven scenarios. IEEE Trans. Visual Comput. Graphics 18(9), 1520–1536 (2012)

    Article  Google Scholar 

  21. Laugwitz, B., Held, T., Schrepp, M.: Construction and evaluation of a user experience questionnaire. In: Holzinger, A. (ed.) USAB 2008. LNCS, vol. 5298, pp. 63–76. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89350-9_6

    Chapter  Google Scholar 

  22. Lecossier, A., Pallot, M., Crubleau, P., Richir, S.: Construction of an instrument to evaluate the user experience of a group of co-creators in the upstream innovation process. Int. J. Serv. Oper. Inf. 10(1), 17–42 (2019)

    Google Scholar 

  23. Mahyar, N., Kim, S.H., Kwon, B.C.: Towards a taxonomy for evaluating user engagement in information visualization. In: Personal Visualization: Exploring Data in Everyday Life: An IEEE VIS 2015 Workshop, pp. 1–4 (2015)

    Google Scholar 

  24. Mandryk, R.L., Inkpen, K.M., Calvert, T.W.: Using psychophysiological techniques to measure user experience with entertainment technologies. Behav. Inf. Technol. 25(2), 141–158 (2006). https://doi.org/10.1080/01449290500331156

    Article  Google Scholar 

  25. Mazza, R.: Introduction to Information Visualization. Springer, London (2009). https://doi.org/10.1007/978-1-84800-219-7

    Book  Google Scholar 

  26. Minge, M., Thüring, M., Wagner, I., Kuhr, C.V.: The meCUE questionnaire: a modular tool for measuring user experience. In: Soares, M., Falcão, C., Ahram, T. (eds.) Advances in Ergonomics Modeling, Usability & Special Populations, pp. 115–128. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-41685-4_11

    Chapter  Google Scholar 

  27. Mortier, R., Haddadi, H., Henderson, T., McAuley, D., Crowcroft, J.: Human-data interaction: the human face of the data-driven society. SSRN Electron. J. 1, 1–14 (2014)

    Google Scholar 

  28. Norman, D.: The Design of Everyday Things: Revised and Expanded Edition. Basic Books, Philadelphia (2013)

    Google Scholar 

  29. Norman, D.A.: Emotional Design: Why We Love (or Hate) Everyday Things. Basic Civitas Books, New York (2004)

    Google Scholar 

  30. Olson, J.S., Kellogg, W.A.: Ways of Knowing in HCI, vol. 2. Springer, New York (2014). https://doi.org/10.1007/978-1-4939-0378-8

    Book  Google Scholar 

  31. Plaisant, C.: The challenge of information visualization evaluation. In: Proceedings of the Working Conference on Advanced Visual Interfaces, AVI 2004, pp. 109–116. Association for Computing Machinery, New York (2004). https://doi.org/10.1145/989863.989880

  32. Robinson, R.: All the Feels: A Twitch Overlay that Displays Streamers’ Biometrics to Spectators. University of California, Santa Cruz (2018)

    Google Scholar 

  33. Saket, B., Endert, A., Stasko, J.: Beyond usability and performance: a review of user experience-focused evaluations in visualization. In: Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization, pp. 133–142 (2016)

    Google Scholar 

  34. Saket, B., Scheidegger, C., Kobourov, S.: Towards understanding enjoyment and flow in information visualization. In: Eurographics Conference on Visualization (EuroVis) (2015)

    Google Scholar 

  35. Saris, W.E., Gallhofer, I.N.: Design, Evaluation, and Analysis of Questionnaires for Survey Research. Wiley, Hoboken (2014)

    Book  MATH  Google Scholar 

  36. Thüring, M., Mahlke, S.: Usability, aesthetics and emotions in human-technology interaction. Int. J. Psychol. 42(4), 253–264 (2007). https://doi.org/10.1080/00207590701396674

    Article  Google Scholar 

  37. van Willigen, T.: Measuring the user experience of data visualization. Master’s thesis, University of Twente (2019)

    Google Scholar 

  38. Väänänen-Vainio-Mattila, K., Segerståhl, K.: A tool for evaluating service user experience (servux): development of a modular questionnaire. In: Interact 2009 conference, User Experience Evaluation Methods in Product Development (UXEM 2009), Workshop in Interact 2009 Conference, Uppsala, Sweden, 2009. p. 4 (2009)

    Google Scholar 

  39. Väätäjä, H., Koponen, T., Roto, V.: Develo** practical tools for user experience evaluation: a case from mobile news journalism. In: European Conference on Cognitive Ergonomics: Designing Beyond the Product-Understanding Activity and User Experience in Ubiquitous Environments, pp. 1–8 (2009)

    Google Scholar 

  40. Victorelli, E.Z., Reis, J.C.D.: Human-data interaction design guidelines for visualization systems. In: Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems. IHC 2020. Association for Computing Machinery (2020). https://doi.org/10.1145/3424953.3426511

  41. Vissers, J., De Bot, L., Zaman, B.: Memoline: evaluating long-term UX with children. In: Proceedings of the 12th International Conference on Interaction Design and Children, pp. 285–288 (2013)

    Google Scholar 

  42. Yi, J.S., Kang, Y., Stasko, J.T., Jacko, J.A.: Toward a deeper understanding of the role of interaction in information visualization. IEEE Trans. Visual Comput. Graphics 13(6), 1224–1231 (2007). https://doi.org/10.1109/TVCG.2007.70515

    Article  Google Scholar 

Download references

Acknowledgements

We want to thank all students for participating in the research and Andressa Santos for her support in conducting the activities. This work was partially supported by the São Paulo Research Foundation (FAPESP) (grants #2013/08293-7, #2015/16528-0, and #2022/15816-5) (The opinions expressed in here are not necessarily shared by the financial support agency).

Author information

Authors and Affiliations

  1. Institute of Computing and Nucleus of Informatics Applied to Education, University of Campinas, Campinas, Brazil

    Eliane Zambon Victorelli & Julio Cesar dos Reis

Authors
  1. Eliane Zambon Victorelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Julio Cesar dos Reis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eliane Zambon Victorelli .

Editor information

Editors and Affiliations

  1. Tokyo City University, Tokyo, Japan

    Hirohiko Mori

  2. Tokyo University of Science, Tokyo, Japan

    Yumi Asahi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Victorelli, E.Z., dos Reis, J.C. (2023). Evaluating User Experience in Information Visualization Systems: UXIV an Evaluation Questionnaire. In: Mori, H., Asahi, Y. (eds) Human Interface and the Management of Information. HCII 2023. Lecture Notes in Computer Science, vol 14015. Springer, Cham. https://doi.org/10.1007/978-3-031-35132-7_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35132-7_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35131-0

  • Online ISBN: 978-3-031-35132-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation