SpringerLink
Log in

Digital Modeling for Everyone: Exploring How Novices Approach Voice-Based 3D Modeling

  • Conference paper
  • First Online:
Human-Computer Interaction – INTERACT 2023 (INTERACT 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14145))

Included in the following conference series:

Abstract

Manufacturing tools like 3D printers have become accessible to the wider society, making the promise of digital fabrication for everyone seemingly reachable. While the actual manufacturing process is largely automated today, users still require knowledge of complex design applications to produce ready-designed objects and adapt them to their needs or design new objects from scratch. To lower the barrier to the design and customization of personalized 3D models, we explored novice mental models in voice-based 3D modeling by conducting a high-fidelity Wizard of Oz study with 22 participants. We performed a thematic analysis of the collected data to understand how the mental model of novices translates into voice-based 3D modeling. We conclude with design implications for voice assistants. For example, they have to: deal with vague, incomplete and wrong commands; provide a set of straightforward commands to shape simple and composite objects; and offer different strategies to select 3D objects.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://zoom.us.

  2. 2.

    https://www.blender.org.

  3. 3.

    https://docs.blender.org/api/current/.

  4. 4.

    https://obsproject.com.

  5. 5.

    https://shorturl.at/fGLRZ.

References

  1. Billinghurst, M., Baldis, S., Matheson, L., Philips, M.: 3d palette: a virtual reality content creation tool. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST 1997, pp. 155–156. Association for Computing Machinery, Lausanne, Switzerland (1997). https://doi.org/10.1145/261135.261163

  2. Bonner, S.E.: A model of the effects of audit task complexity. Acc. Organ. Soc. 19(3), 213–234 (1994). https://doi.org/10.1016/0361-3682(94)90033-7

    Article  Google Scholar 

  3. Braun, M., Mainz, A., Chadowitz, R., Pfleging, B., Alt, F.: At your service: designing voice assistant personalities to improve automotive user interfaces. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, pp. 1–11. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3290605.3300270

  4. Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3(2), 77–101 (2006). https://doi.org/10.1191/1478088706qp063oa

    Article  Google Scholar 

  5. Cuadra, A., Goedicke, D., Zamfirescu-Pereira, J.: Democratizing design and fabrication using speech: Exploring co-design with a voice assistant. In: CUI 2021–3rd Conference on Conversational User Interfaces, CUI 2021, Association for Computing Machinery, Bilbao (online), Spain (2021). https://doi.org/10.1145/3469595.3469624

  6. Cuadra, A., Li, S., Lee, H., Cho, J., Ju, W.: My bad! repairing intelligent voice assistant errors improves interaction. Proc. ACM Hum.-Comput. Interact. 5(CSCW1) (2021). https://doi.org/10.1145/3449101

  7. Davis, K.H., Biddulph, R., Balashek, S.: Automatic recognition of spoken digits. J. Acoustical Soc. Am. 24(6), 637–642 (1952). https://doi.org/10.1121/1.1906946

    Article  Google Scholar 

  8. Etikan, I., Musa, S.A., Alkassim, R.S.: Comparison of convenience sampling and purposive sampling. Am. J. Theoret. Appli. Stat. 5(1), 1–4 (2016). https://doi.org/10.11648/j.ajtas.20160501.11

  9. Fast, E., Chen, B., Mendelsohn, J., Bassen, J., Bernstein, M.S.: Iris: a conversational agent for complex tasks. CoRR abs/ ar**v: 1707.05015 (2017)

  10. Feeman, S.M., Wright, L.B., Salmon, J.L.: Exploration and evaluation of cad modeling in virtual reality. Comput.-Aided Design Appli. 15(6), 892–904 (2018). https://doi.org/10.1080/16864360.2018.1462570

    Article  Google Scholar 

  11. Fialho, P., Coheur, L.: Chatwoz: chatting through a wizard of oz. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, ASSETS 2015, pp. 423–424. , Association for Computing Machinery, Lisbon, Portugal (2015). https://doi.org/10.1145/2700648.2811334

  12. Friedrich, M., Langer, S., Frey, F.: Combining gesture and voice control for mid-air manipulation of cad models in vr environments. In: Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, vol. 1: HUCAPP, pp. 119–127. INSTICC, SciTePress, Online Streaming (2021). https://doi.org/10.5220/0010170501190127

  13. Gershenfeld, N.: How to make almost anything: the digital fabrication revolution. Foreign Aff. 91, 43 (2012)

    Google Scholar 

  14. Grigor, S., Nandra, C., Gorgan, D.: Voice-controlled 3d modelling with an intelligent personal assistant. Int. J. User-Syst. Interaction 13, 73–88 (2020). https://doi.org/10.37789/ijusi.2020.13.2.2

  15. Grimes, G.M., Schuetzler, R.M., Giboney, J.S.: Mental models and expectation violations in conversational AI interactions. Decis. Support Syst. 144, 113515 (2021). https://doi.org/10.1016/j.dss.2021.113515

    Article  Google Scholar 

  16. van Gumster, J.: Blender For Dummies, 2nd edn. For Dummies, USA (2011)

    Google Scholar 

  17. Huang, Y.-C., Chen, K.-L.: Brain-computer interfaces (BCI) based 3D computer-aided design (CAD): to improve the efficiency of 3D modeling for new users. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2017. LNCS (LNAI), vol. 10285, pp. 333–344. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58625-0_24

    Chapter  Google Scholar 

  18. Jain, M., Kumar, P., Kota, R., Patel, S.N.: Evaluating and informing the design of chatbots. In: Proceedings of the 2018 Designing Interactive Systems Conference, DIS 2018, pp. 895–906. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3196709.3196735

  19. James, J., Watson, C.I., MacDonald, B.: Artificial empathy in social robots: an analysis of emotions in speech. In: Proceedings of the 27th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2018, pp. 632–637. IEEE Press, Nan**g, China (2018). https://doi.org/10.1109/ROMAN.2018.8525652

  20. Jowers, I., Prats, M., McKay, A., Garner, S.: Evaluating an eye tracking interface for a two-dimensional sketch editor. Comput. Aided Des. 45(5), 923–936 (2013). https://doi.org/10.1016/j.cad.2013.01.006

    Article  Google Scholar 

  21. Khan, S., Tunçer, B.: Gesture and speech elicitation for 3d cad modeling in conceptual design. Autom. Constr. 106, 102847 (2019). https://doi.org/10.1016/j.autcon.2019.102847

    Article  Google Scholar 

  22. Khan, S., Tuncer, B., Subramanian, R., Blessing, L.: 3d cad modeling using gestures and speech: investigating cad legacy and non-legacy procedures. In: Lee, J.H. (ed.) Proceedings of the 18th International Conference on Computer Aided Architectural Design Futures, CAAD Futures 2019, pp. 347–366. CUMINCAD, Daejeon, Republic of Korea (2019). http://papers.cumincad.org/cgi-bin/works/paper/cf2019_042

  23. Kou, X.Y., Tan, S.T.: Design by talking with computers. Comput.-Aided Design Appli. 5(1–4), 266–277 (2008). https://doi.org/10.3722/cadaps.2008.266-277

    Article  Google Scholar 

  24. Kou, X., Xue, S., Tan, S.: Knowledge-guided inference for voice-enabled cad. Comput. Aided Des. 42(6), 545–557 (2010). https://doi.org/10.1016/j.cad.2010.02.002

    Article  Google Scholar 

  25. Langevin, R., Lordon, R.J., Avrahami, T., Cowan, B.R., Hirsch, T., Hsieh, G.: Heuristic evaluation of conversational agents. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI 2021. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3411764.3445312

  26. Lee, M., Billinghurst, M.: A wizard of oz study for an ar multimodal interface. In: Proceedings of the 10th International Conference on Multimodal Interfaces, ICMI 2008, pp. 249–256. Association for Computing Machinery, Chania, Crete, Greece (2008). https://doi.org/10.1145/1452392.1452444

  27. Mayer, S., Laput, G., Harrison, C.: Enhancing mobile voice assistants with worldgaze. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI 2020, pp. 1–10. Association for Computing Machinery, Honolulu, HI, USA (2020). https://doi.org/10.1145/3313831.3376479

  28. Medhi Thies, I., Menon, N., Magapu, S., Subramony, M., O’Neill, J.: How do you want your Chatbot? an exploratory Wizard-of-Oz study with young, Urban Indians. In: Bernhaupt, R., Dalvi, G., Joshi, A., Balkrishan, D.K., O’Neill, J., Winckler, M. (eds.) INTERACT 2017. LNCS, vol. 10513, pp. 441–459. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67744-6_28

    Chapter  Google Scholar 

  29. Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995). https://doi.org/10.1145/219717.219748

    Article  Google Scholar 

  30. Moore, R.J., Arar, R.: Conversational UX Design: A Practitioner’s Guide to the Natural Conversation Framework. Association for Computing Machinery, New York (2019)

    Google Scholar 

  31. Murad, C., Munteanu, C., Cowan, B.R., Clark, L.: Revolution or evolution? speech interaction and HCI design guidelines. IEEE Pervasive Comput. 18(2), 33–45 (2019). https://doi.org/10.1109/MPRV.2019.2906991

    Article  Google Scholar 

  32. Murad, C., Munteanu, C., Cowan, B.R., Clark, L.: Finding a new voice: transitioning designers from gui to vui design. In: CUI 2021–3rd Conference on Conversational User Interfaces, CUI 2021. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3469595.3469617

  33. Nanjundaswamy, V.G., et al.: Intuitive 3D computer-aided design (CAD) system with multimodal interfaces. In: Proceedings of the ASME 2013 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, vol. 2A: 33rd Computers and Information in Engineering Conference. ASME, Portland, Oregon, USA (08 2013). https://doi.org/10.1115/DETC2013-12277, v02AT02A037

  34. Nielsen, J.: 10 usability heuristics for user interface design (1994). https://www.nngroup.com/articles/ten-usability-heuristics/

  35. Nielsen, J.: Enhancing the explanatory power of usability heuristics. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 1994, pp. 152–158. Association for Computing Machinery, Boston, Massachusetts, USA (1994). https://doi.org/10.1145/191666.191729

  36. Niu, H., Van Leeuwen, C., Hao, J., Wang, G., Lachmann, T.: Multimodal natural human-computer interfaces for computer-aided design: A review paper. Appl. Sci. 12(13), 6510 (2022)

    Article  Google Scholar 

  37. Nowacki, C., Gordeeva, A., Lizé, A.H.: Improving the usability of voice user interfaces: a new set of ergonomic criteria. In: Marcus, A., Rosenzweig, E. (eds.) Design, User Experience, and Usability. Design for Contemporary Interactive Environments, pp. 117–133. Springer International Publishing, Cham (2020)

    Google Scholar 

  38. Plumed, R., González-Lluch, C., Pérez-López, D., Contero, M., Camba, J.D.: A voice-based annotation system for collaborative computer-aided design. J. Comput. Design Eng. 8(2), 536–546 (2021). https://doi.org/10.1093/jcde/qwaa092

    Article  Google Scholar 

  39. Sabharwal, N., Agrawal, A.: Introduction to Google Dialogflow, pp. 13–54. Apress, Berkeley, CA (2020). https://doi.org/10.1007/978-1-4842-5741-8_2

  40. Samad, T., Director, S.W.: Towards a natural language interface for cad. In: Proceedings of the 22nd ACM/IEEE Design Automation Conference, pp. 2–8. DAC ’85, IEEE Press, Las Vegas, Nevada, USA (1985)

    Google Scholar 

  41. Seaborn, K., Miyake, N.P., Pennefather, P., Otake-Matsuura, M.: Voice in human-agent interaction: a survey. ACM Comput. Surv. (CSUR) 54(4), 1–43 (2021)

    Article  Google Scholar 

  42. Sharp, H., Rogers, Y., Preece, J.: Interaction Design: Beyond Human Computer Interaction. John Wiley & Sons Inc., Hoboken, NJ, USA (2007)

    Google Scholar 

  43. Shum, H.Y., Han, M., Szeliski, R.: Interactive construction of 3d models from panoramic mosaics. In: Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231), pp. 427–433. IEEE, Santa Barbara, California (1998). https://doi.org/10.1109/CVPR.1998.698641

  44. Sree Shankar, S., Rai, R.: Human factors study on the usage of BCI headset for 3d cad modeling. Comput.-Aided Design 54, 51–55 (2014). https://doi.org/10.1016/j.cad.2014.01.006, application of brain-computer interfaces in cad/e systems

  45. Stark, R., Israel, J., Wöhler, T.: Towards hybrid modelling environments–merging desktop-cad and virtual reality-technologies. CIRP Ann. 59(1), 179–182 (2010). https://doi.org/10.1016/j.cirp.2010.03.102

    Article  Google Scholar 

  46. Thakur, A., Rai, R.: User study of hand gestures for gesture based 3D CAD modeling. In: Proceedings of the ASME 2015 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, vol. 1B: 35th Computers and Information in Engineering Conference. ASME, Boston, Massachusetts, USA (Aug 2015). https://doi.org/10.1115/DETC2015-46086, v01BT02A017

  47. Voice2CAD: Voice2CAD (2022). https://voice2cad.com/

  48. Völkel, S.T., Buschek, D., Eiband, M., Cowan, B.R., Hussmann, H.: Eliciting and analysing users’ envisioned dialogues with perfect voice assistants. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI 2021. Association for Computing Machinery, Yokohama, Japan (2021). https://doi.org/10.1145/3411764.3445536

  49. Vtyurina, A., Fourney, A.: Exploring the role of conversational cues in guided task support with virtual assistants. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, pp. 1–7. Association for Computing Machinery, Montreal QC, Canada (2018). https://doi.org/10.1145/3173574.3173782

  50. Vuletic, T., et al.: A novel user-based gesture vocabulary for conceptual design. Int. J. Hum Comput Stud. 150, 102609 (2021). https://doi.org/10.1016/j.ijhcs.2021.102609

    Article  Google Scholar 

  51. Vyas, S., Chen, T.J., Mohanty, R., Krishnamurthy, V.: Making-a-scene: a preliminary case study on speech-based 3D shape exploration through scene modeling. J. Comput. Inform. Sci. Eng. 1–11 (2022). https://doi.org/10.1115/1.4055239

  52. Xue, S., Kou, X., Tan, S.: Natural voice-enabled cad: modeling via natural discourse. Comput.-Aided Design Appli. 6(1), 125–136 (2009)

    Article  Google Scholar 

  53. Xue, S., Kou, X., Tan, S.: Command search for CAD system. Comput.-Aided Design Appli. 7(6), 899–910 (2010)

    Article  Google Scholar 

  54. Yoon, S.M., Graf, H.: Eye tracking based interaction with 3d reconstructed objects. In: Proceedings of the 16th ACM International Conference on Multimedia, MM 2008, pp. 841–844. Association for Computing Machinery, Vancouver, British Columbia, Canada (2008). https://doi.org/10.1145/1459359.1459501

Download references

Acknowledgements

This work has been funded by the European Union’s Horizon 2020 research and innovation program under grant agreement No. 952026 (https://www.humane-ai.eu/). The research of Andrea Esposito is funded by a Ph.D. fellowship within the framework of the Italian “D.M. n. 352, April 9, 2022’ - under the National Recovery and Resilience Plan, Mission 4, Component 2, Investment 3.3 - Ph.D. Project “Human-Centered Artificial Intelligence (HCAI) techniques for supporting end users interacting with AI systems”, co-supported by “Eusoft S.r.l.” (CUP H91I22000410007).

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Bari Aldo Moro, Bari, Italy

    Giuseppe Desolda & Andrea Esposito

  2. LMU Munich, Munich, Germany

    Florian Müller & Sebastian Feger

Authors
  1. Giuseppe Desolda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Esposito
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Florian Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sebastian Feger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giuseppe Desolda .

Editor information

Editors and Affiliations

  1. University of West London, London, UK

    José Abdelnour Nocera

  2. Reykjavik University, Reykjavik, Iceland

    Marta Kristín Lárusdóttir

  3. University of York, York, UK

    Helen Petrie

  4. University of Bari Aldo Moro, Bari, Italy

    Antonio Piccinno

  5. Université Côte d’Azur, Sophia Antipolis Cedex, France

    Marco Winckler

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (zip 129 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Desolda, G., Esposito, A., Müller, F., Feger, S. (2023). Digital Modeling for Everyone: Exploring How Novices Approach Voice-Based 3D Modeling. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. Lecture Notes in Computer Science, vol 14145. Springer, Cham. https://doi.org/10.1007/978-3-031-42293-5_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42293-5_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42292-8

  • Online ISBN: 978-3-031-42293-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation