SpringerLink
Log in

Towards a Practice-Led Research Agenda for User Interface Design of Recommender Systems

  • Conference paper
  • First Online:
Human-Computer Interaction – INTERACT 2023 (INTERACT 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14144))

Included in the following conference series:

Abstract

The design of recommender systems’ graphical user interfaces (GUIs) is critical for a user's experience with these systems. However, most research into recommenders focuses on algorithms, overlooking the design of their interfaces. Additionally, the studies on the design of recommender interfaces that do exist do not always manage to cross the research-practice gap. This disconnect may be due to a lack of alignment between academic focus and the most pressing needs of practitioners, as well as the way research findings are communicated. To address these issues, this paper presents the results of a comprehensive study involving 215 designers worldwide, aiming to identify the primary challenges in designing recommender GUIs and the resources practitioners need to tackle those challenges. Building on these findings, this paper proposes a practice-led research agenda for the human-computer interaction community on designing recommender interfaces and suggestions for more accessible and actionable ways of disseminating research results in this domain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Germany)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 74.89
Price includes VAT (Germany)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 96.29
Price includes VAT (Germany)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Resnick, P., Varian, H.R: Recommender Systems. Communications of the ACM 40(3), 56–58 (1997). https://doi.org/10.1145/245108.245121

  2. Beel, J., Dixon, H.M: The ‘unreasonable’ effectiveness of graphical user interfaces for recommender systems. In: 2021 Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, pp. 22–28 (2021). https://doi.org/10.1145/3450614.3461682

  3. Gunawardana, A., Shani, G., Yogev, S.: Evaluating recommender systems. In: Ricci, F., Rokach, L., Shapira, B. (eds) Recommender Systems Handbook, pp. 547–601. Springer, New York, NY (2022). https://doi.org/10.1007/978-1-0716-2197-4_15

  4. Jugovac, M., Jannach, D.: Interacting with recommenders—overview and research directions. In: Zhou, M., (ed) ACM Transactions on Interactive Intelligent Systems, 7(3), pp. 1–46 (2017). Association for Computing Machinery, New York, NY. https://doi.org/10.1145/3001837

  5. Ghori, M., Dehpanah, A., Gemmell, J., Qahri-Saremi, H., Mobasher, B.: Does the user have a theory of the recommender? a grounded theory study. In: 2021 Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization, pp. 167–174. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3511047.3537680

  6. Holmquist, L.: Intelligence on tap: artificial intelligence as a new design material. Interactions 24(4), 28–33 (2017). https://doi.org/10.1145/3085571

    Article  MathSciNet  Google Scholar 

  7. Goodman, E., Stolterman, E., Wakkary, R.: Understanding interaction design practices. In: Proceedings of the 2011 SIGCHI Conference on Human Factors in Computing Systems, pp. 1061–1070. Association for Computing Machinery, New York (2011). https://doi.org/10.1145/1978942.1979100

  8. Dove, G., Halskov, K., Forlizzi, J., Zimmerman, J.: UX design innovation: challenges for working with machine learning as a design material. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 278–288. Association for Computing Machinery, New York (2017). https://doi.org/10.1145/3025453.3025739

  9. Yang, Q., Scuito, A., Zimmerman, J., Forlizzi, J., Steinfeld, A.: Investigating how experienced UX designers effectively work with machine learning. In: Proceedings of the 2018 Designing Interactive Systems Conference, pp. 585–596. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3196709.3196730

  10. Hekman, E., Nguyen, D., Stalenhoef, M., Van Turnhout, K.: Towards a pattern library for algorithmic affordances. In: Joint Proceedings of the IUI 2022 Workshops, 3124, pp. 24–33. (2022). https://ceur-ws.org/Vol-3124/paper3.pdf

  11. Knijnenburg, B.P., Willemsen, M.C., Gantner, Z., Soncu, H., Newell, C.: Explaining the user experience of recommender systems. User Model. User-Adap. Inter. 22(4), 441–504 (2012). https://doi.org/10.1007/s11257-011-9118-4

    Article  Google Scholar 

  12. Pu, P., Chen, L., Hu, R.: A user-centric evaluation framework for recommender systems. In: Proceedings of the Fifth ACM Conference on Recommender Systems 2011, pp. 157–164. Association for Computing Machinery, New York (2011). https://doi.org/10.1145/2043932.2043962

  13. Murphy-Hill, E., Murphy, G.: Recommendation delivery: Getting the user interface just right. In Robillard, M., Maalej, W., Walker, R., Zimmerman, T. (eds) Recommendation systems in software engineering, pp. 223–242. Springer, Berlin, Heidelberg (2014). https://doi.org/10.1007/978-3-642-45135-5_9

  14. Swearingen, K., Sinha, R.: Interaction design for recommender systems. In Designing Interactive Systems 6(12), 312–334 (2002)

    Google Scholar 

  15. Rogers, Y.: New theoretical approaches for human-computer interaction. In: Cronin, B. (ed) Annual Review of Information Science and Technol. 38(1), 87–143. Wiley, Hoboken (NJ) (2005). https://doi.org/10.1002/aris.1440380103

  16. Zielhuis, M., Visser, F., Andriessen, D., Stappers, P.: What makes design research more useful for design professionals? an exploration of the research-practice gap. J. Design Res. 20(2), 105–122 (2022)

    Article  Google Scholar 

  17. Buie, E., Hooper, C., Houssian, A.: Research-practice interaction: building bridges, closing the gap. In: CHI 2013 Extended Abstracts on Factors in Computing Systems, pp. 13–16. Association for Computing Machinery, New York (2013). https://doi.org/10.1145/2468356.2468813

  18. Löwgren, J.: Annotated portfolios and other forms of intermediate-level knowledge. Interactions 20(1), 30–34 (2013). https://doi.org/10.1145/2405716.2405725

    Article  Google Scholar 

  19. Gaver, B., Bowers, J.: Annotated portfolios. Interactions 19(4), 40–49 (2012). https://doi.org/10.1145/2212877.2212889

    Article  Google Scholar 

  20. Norman, D.: The research-practice gap: the need for translational developers. Interactions 17(4), 9–12 (2010). https://doi.org/10.1145/1806491.1806494

    Article  Google Scholar 

  21. Parsons, P., Shukla, P.: Considering the Role of Guidelines in Visualization Design Practice. https://doi.org/10.31219/osf.io/mw376

  22. Zielhuis, M.: Discomfort as a starting point: how design research can contribute to design practice. In: Joore, P., Stompff, G., Van den Eijnde, J. (eds.) Applied Design Research, pp. 114–123. CRC Press, Boca Raton (2022)

    Google Scholar 

  23. Colusso, L., Bennet, C., Hsieh, G., Munson, S.: Translational resources: reducing the gap between academic research and HCI practice. In: Proceedings of the 2017 Conference on Designing Interactive Systems, pp. 957–968 (2017). https://doi.org/10.1145/3064663.3064667

  24. Kou, Y., Gray, C.: A practice-led account of the conceptual evolution of UX knowledge. In: CHI 2019 Conference on Human Factors in Computing Systems, pp. 1–13. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3290605.3300279

  25. Bolin, E., Gray C.: Use of precedent as a narrative practice in design learning. In: Hokanson, B., Clinton, G., Kaminski, K. (eds) Educational Technology and Narrative: Story and Instructional Design, pp. 259–270. Springer Cham, New York (2017). https://doi.org/10.1007/978-3-319-69914-1_21

  26. Yang, Q., Banovic, N., Zimmerman, J.: Map** machine learning advances from HCI research to reveal starting places for design research. In: CHI Conference on Human Factors in Computing Systems, pp. 1–11. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3173574.3173704

  27. Turnhout, K., Smits, A.: Solution repertoire. In: Grierson, H., Bohemia, R., Buck, L. (eds) Proceedings of the 23rd International Conference on Engineering and Product Design Education (2021). https://doi.org/10.35199/EPDE.2021.41

  28. Smits, A., Van Turnhout, K., Hekman, E., Nguyen, D.: Data-driven design. In: Buck, L., Bohemia, E., Grierson, H. (eds) Proceedings of the 22nd International Conference on Engineering and Product Design Education (2020). https://doi.org/10.35199/EPDE.2020.10

  29. Konstan, J., Riedl, J.: Recommender systems: from algorithms to user experience. User Model. User-Adap. Inter. 22(1), 101–123 (2012). https://doi.org/10.1007/s11257-011-9112-x

    Article  Google Scholar 

  30. Iaquinta, L., Gemmis, M., Lops, P., Semeraro, G.: Introducing serendipity in a content-based recommender system. In: 8th International Conference on Hybrid Intelligent Systems, pp. 168–174. IEEE, New York (2008). https://doi.org/10.1109/HIS.2008.25

  31. Kamahara, J., Asakawa, T., Shimojo, S., Miyahara, H.: A community-based recommendation system to reveal unexpected interests. In: Chen, Y. (ed) 11th International Multimedia Modeling Conference 2005, pp. 433–438. IEEE, New York (2005). https://doi.org/10.1109/MMMC.2005.5

  32. Schnabel, T., Bennet, T. Joachim, T.: Improving Recommender Systems Beyond the Algorithm (2018). https://doi.org/10.48550/ar**v.1802.07578

  33. Tintarev, N., Masthoff, J: Explaining recommendations: design and evaluation. In: Ricci, F., Rocach, L., Shapira, B. (eds) Recommender systems handbook, pp. 353–382. Springer, New York (2015). https://doi.org/10.1007/978-1-4899-7637-6_10

  34. Verbert, K., Parra, D., Brusilovsky, P., Duval, E.: Visualizing recommendations to support exploration, transparency, and controllability. In: Proceedings of the 2013 international conference on Intelligent user interfaces, pp. 351–362. Association for Computing Machinery, New York (2013). https://doi.org/10.1145/2449396.2449442

  35. Pasquale, F.: The Black Box Society. Harvard University Press, Cambridge, MA (2015)

    Book  Google Scholar 

  36. Schoonderwoerd, T., Jorritsma, W., Neerincx, M., Van den Bosch, K.: Human-centered XAI: Develo** design patterns for explanations of clinical decision support systems. International Journal of Human-Computer Studies 54, (2021). https://doi.org/10.1016/j.ijhcs.2021.102684

  37. Champiri, Z., Mujtaba, G., Salim, S., Chong, C.: User experience and recommender systems. In: 2nd International Conference on Computing, Mathematics and Engineering Technologies (2019), pp. 1–5. IEEE, New York (2019). https://doi.org/10.1109/ICOMET.2019.8673410

  38. Xu, F., Uszkoreit, H., Du, Y., Fan, W., Zhao, D., Zhu, J.: Explainable AI: a brief survey on history, research areas, approaches and challenges. In Natural Language Processing and Chinese Computing: 8th CCF International Conference, NLPCC 2019, Dunhuang, China, October 9–14, 2019, Proceedings, Part II 8, pp. 563–574. Springer International Publishing (2019). https://doi.org/10.1007/978-3-030-32236-6_51

  39. Liao, Q.V., Gruen, D., Miller, S.: Questioning the AI: informing design practices for explainable AI user experiences. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–15 (2020). https://doi.org/10.1145/3313831.3376590

  40. Ozok, A., Fan, Q., Norcio, A.: Design guidelines for effective recommender system interfaces based on a usability criteria conceptual model: results from a college student population. Behaviour & Inf. Technol. 29(1), 57–83 (2010). https://doi.org/10.1080/01449290903004012

    Article  Google Scholar 

  41. Zangerle, E., Bauer, C.: Evaluating recommender systems: survey and framework. ACM Comput. Surv. 55(8), 1–38 (2022). https://doi.org/10.1145/3556536

    Article  Google Scholar 

  42. Herlocker, J., Konstan, J., Riedl, J.: Explaining collaborative filtering recommendations. In: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, pp. 241–250. Association for Computing Machinery, New York (2000). https://doi.org/10.1145/358916.358995

  43. Tsai, C., Brusilovsky, P.: Designing explanation interfaces for transparency and beyond. In: Joint ACM IUI Workshops (2019), p. 2327 (2017). https://ceur-ws.org/Vol-2327/IUI19WS-IUIATEC-4.pdf

  44. Chen, L., Tsoi, H.: Users’ decision behavior in recommender interfaces: impact of layout design. In: RecSys 2011 Workshop on Human Decision Making in Recommender Systems (2011). https://ceur-ws.org/Vol-811/paper4.pdf

  45. Jannach, D., Jesse, M., Jugovac, M., Trattner, C.: Exploring multi-list user interfaces for similar-item recommendations. In: Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, pp. 224–228. Association for Computing Machinery, New York, NY (2021). https://doi.org/10.1145/3450613.3456809

  46. Chen, L., Pu, P.: Eye-tracking study of user behavior in recommender interfaces. In: Bra, P., Kobsa, A., Chin, D. (eds) User Modeling, Adaptation, and Personalization, pp. 375–380. Springer, Berlin, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13470-8_35

  47. Kammerer, Y., Gerjets, P.: The role of search result position and source trustworthiness in the selection of web search results when using a list or a grid interface. International J. Human-Computer Interaction 30(3), 177–191 (2014). https://doi.org/10.1080/10447318.2013.846790

    Article  Google Scholar 

  48. Starke, A., Asotic, E., Trattner, C.: Serving each user: supporting different eating goals through a multi-list recommender interface. In: Fifteenth ACM Conference on Recommender Systems, pp. 124–132. Association for Computing Machinery, New York, NY (2021). https://doi.org/10.1145/3460231.3474232

  49. Eslami, M., et al.: First i” like” it, then i hide it: folk theories of social feeds. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 2371–2382. Association for Computing Machinery, New York, NY (2016). https://doi.org/10.1145/2858036.2858494

  50. Ngo, T., Kunkel, J., Ziegler, J.: Exploring mental models for transparent and controllable recommender systems: a qualitative study. In: Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, pp. 183–191. Association for Computing Machinery, New York, NY (2020). https://doi.org/10.1145/3340631.3394841

  51. Nielsen, J.: Putting A/B testing in its place. Nielsen Norman Group. https://www.nngroup.com/articles/putting-ab-testing-in-its-place/, Accessed 29 Mar 2023

  52. Salminen, J., Kandpal, C., Kamel, A., Jung, S., Jansen, B.: Creating and detecting fake reviews of online products. Journal of Retailing and Consumer Services 64, 102771 (2022). https://doi.org/10.1016/j.jretconser.2021.102771

  53. Cremonesi, P., Elahi, M., Garzotto, F.: User interface patterns in recommendation-empowered content intensive multimedia applications. Multimed Tools Appl. 76, 5275–5309 (2017). https://doi.org/10.1007/s11042-016-3946-5

    Article  Google Scholar 

  54. Nunes, I., Jannach, D.: A systematic review and taxonomy of explanations in decision support and recommender systems. User Model. User-Adap. Inter. 27(3), 393–444 (2017). https://doi.org/10.1007/s11257-017-9195-0

    Article  Google Scholar 

  55. Dietvorst, B., Simmons, J., Massey, C.: Overcoming algorithm aversion: people will use imperfect algorithms if they can (even slightly) modify them. Manage. Sci. 64(3), 1155–1170 (2018). https://doi.org/10.1287/mnsc.2016.2643

    Article  Google Scholar 

  56. Creswell, J., Clark, V.: Designing and Conducting Mixed Methods Research. Sage, Thousand Oaks (2009)

    Google Scholar 

  57. Gray, C.: It's More of a Mindset Than a Method: UX practitioners’ conception of design methods. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 4044–4055. Association for Computing Machinery, New York, NY (2016). https://doi.org/10.1145/2858036.2858410

  58. Algorithmic Affordances Pattern Library (rudimentary version). https://algorithmicaffordances.com/. Accessed 29 Mar 2023

  59. Smits, A., Bartels, E., Detweiler, C., Van Turnhout, K.: Algorithmic affordances in recommender interfaces. In: Human-Computer Interaction–INTERACT 2023: 19th IFIP TC 13 International Conference, York, UK, August 28–September 1, 2023, Proceedings, Part V 18 (2023)

    Google Scholar 

  60. Zielhuis, M., Visser, F., Andriessen, D., Stappers, P.: Making design research relevant for design practice: what is in the way? Des. Stud. 78(101063), 1–21 (2022). https://doi.org/10.1016/j.destud.2021.101063

    Article  Google Scholar 

Download references

Acknowledgments

We are grateful to the Amsterdam UX meetup community, notably Tatiana Siderenkowa, for hel** us recruit so many committed respondents. Credits for the graphics in Figure 1 and Figure 3 go to Karine Cardona and Ester Bartels.

Author information

Authors and Affiliations

  1. HU University of Applied Sciences, Utrecht, The Netherlands

    Aletta Smits & Koen van Turnhout

Authors
  1. Aletta Smits
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Koen van Turnhout
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aletta Smits .

Editor information

Editors and Affiliations

  1. University of West London, London, UK

    José Abdelnour Nocera

  2. Reykjavik University, Reykjavik, Iceland

    Marta Kristín Lárusdóttir

  3. University of York, York, UK

    Helen Petrie

  4. University of Bari Aldo Moro, Bari, Italy

    Antonio Piccinno

  5. Université Côte d’Azur, Sophia Antipolis Cedex, France

    Marco Winckler

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Smits, A., van Turnhout, K. (2023). Towards a Practice-Led Research Agenda for User Interface Design of Recommender Systems. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. Lecture Notes in Computer Science, vol 14144. Springer, Cham. https://doi.org/10.1007/978-3-031-42286-7_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42286-7_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42285-0

  • Online ISBN: 978-3-031-42286-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation