SpringerLink
Log in

Multisensory Interaction in eXtended Reality

  • Chapter
  • First Online:
Prototy** User eXperience in eXtended Reality

Part of the book series: SpringerBriefs in Applied Sciences and Technology ((BRIEFSPOLIMI))

  • 420 Accesses

Abstract

This chapter provides an overview of eXtended Reality (XR), a term that encompasses technologies such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). XR allows the real world to be enriched with virtual data, objects, and information, offering varying levels of immersion and interaction. The chapter explores the components of XR and discusses classification frameworks, including Milgram’s Reality-Virtuality Continuum and the 3I model. It further delves into the developments and technologies of VR, highlighting stereoscopic vision, VR headsets and immersive walls. The chapter also explores AR, its accessibility through everyday devices, and its applications. The importance of considering all sensory modalities for creating immersive XR experiences is emphasized, and the role of multimodal interaction in enhancing user experiences is discussed. The chapter highlights the significance of a multisensory approach in achieving a sense of presence and explores examples of early immersive and multisensory technology. Finally, it discusses the impact of touch in human interaction and the incorporation of tactile sensations in VR applications, as well as of olfaction, which plays a significant role in human perception and memory, as certain smells can evoke strong emotional responses and trigger vivid recollections of past experiences. Incorporating odors into XR applications can enhance the overall sense of presence and realism by providing users with scent cues that align with the virtual environment. Olfactory displays are devices that release specific odors or scents to accompany virtual content.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Germany)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 37.44
Price includes VAT (Germany)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 48.14
Price includes VAT (Germany)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Milgram P, Takemura H, Utsumi A, Kishino F (1995) Augmented reality: a class of displays on the reality-virtuality continuum. In: Proceedings of SPIE 2351, telemanipulator and telepresence technologies

    Google Scholar 

  2. Burdea G, Coiffet P (2003) Virtual reality technology, 2nd edn. Wiley-Interscience

    Google Scholar 

  3. Moton Heilig. http://www.mortonheilig.com. Last accessed 15 May 2023

  4. Sutherland IE (1968) A head-mounted three dimensional display. In: AFIPS ‘68 (Fall, part I)

    Google Scholar 

  5. Oculus. https://www.oculus.com. Last accessed 15 May 2023

  6. HTC Vive. https://www.vive.com. Last accessed 15 May 2023

  7. Banks MS, Read JC, Allison RS, Watt SJ (2012) Stereoscopy and the human visual system. SMPTE Motion Imaging J 121(4):24–43

    Article  Google Scholar 

  8. Cruz-Neira C, Sandin DJ, DeFanti TA, Kenyon RV, Hart JC (1992) The CAVE: audio visual experience automatic virtual environment. Commun ACM 35(6):64–72

    Article  Google Scholar 

  9. Epson Moverio. https://moverio.epson.com. Last accessed 15 May 2023

  10. Vuzix. https://www.vuzix.com. Last accessed 15 May 2023

  11. Snapchat. https://www.snapchat.com. Last accessed 15 May 2023

  12. Carulli M, Bordegoni M (2019) Multisensory augmented reality experiences for cultural heritage exhibitions. In: Rizzi C, Andrisano A, Leali F, Gherardini F, Pini F, Vergnano A (eds) Design tools and methods in industrial engineering. Lecture notes in mechanical engineering. Springer

    Google Scholar 

  13. Masoni R, Ferrise F, Bordegoni M, Gattullo M, Uva AE, Fiorentino M (2017) Supporting remote maintenance in industry 4.0 through augmented reality. Proc Manuf 11:1296–1302

    Google Scholar 

  14. Microsoft Hololens. https://www.microsoft.com/en-us/hololens. Last accessed 15 May 2023

  15. Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a presence questionnaire. Presence Teleoperators Virtual Environ 7(3):225–240

    Google Scholar 

  16. Steuer J (1992) Defining virtual reality: dimensions determining telepresence. J Commun 42(4):73–93

    Article  Google Scholar 

  17. Burnett S (2011) Perceptual worlds and sensory ecology. Nat Educ Knowl 3(10):75

    Google Scholar 

  18. Bourguet ML (2003) Designing and prototy** multimodal commands. In: Proceedings of human-computer interaction (INTERACT’03), pp 717–720

    Google Scholar 

  19. Marr D (1982) Vision: a computational investigation into the human representation and processing of visual information. Henry Holt and Co Inc., New York, NY

    Google Scholar 

  20. Shams L, Seitz AR (2008) Benefits of multisensory learning. Trends Cogn Sci 12(11):411–417

    Article  Google Scholar 

  21. Spence C, Driver J (2004) Crossmodal space and crossmodal attention. Oxford University Press

    Google Scholar 

  22. Bordegoni M, Ferrise F (2013) Designing interaction with consumer products in a multisensory virtual reality environment. Virtual Phys Prototy** 8(1):51–64

    Article  Google Scholar 

  23. Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a presence questionnaire. Presence 7(3):225–240

    Article  Google Scholar 

  24. Heilig ML (1962) Sensorama simulator, US PAT. 3,050,870

    Google Scholar 

  25. Marto A, Melo M, Gonçalves A, Bessa M (2020) Multisensory augmented reality in cultural heritage: impact of different stimuli on presence, enjoyment, knowledge and value of the experience. IEEE Access 8:193744–193756

    Article  Google Scholar 

  26. Field TM (1995) Touch in early development. Lawrence Erlbaum Associates, Inc.

    Google Scholar 

  27. Gallace A, Spence C (2013) In touch with the future. Oxford University Press

    Google Scholar 

  28. Rolls ET, O’Doherty J, Kringelbach ML, Francis S, Bowtell R, McGlone F (2003) Representations of pleasant and painful touch in the human orbitofrontal and cingulate cortices. Cereb Cortex 13(3):308–317

    Article  Google Scholar 

  29. Etzi R, Gallace A (2016) The arousing power of everyday materials: an analysis of the physiological and behavioral responses to visually and tactually presented textures. Exp Brain Res 234(6):1659–1666

    Article  Google Scholar 

  30. Robles-De-La-Torre G (2006) The importance of the sense of touch in virtual and real environments. IEEE Multim 13(3), Special issue on Haptic User Interfaces for Multimedia Systems, 24–30

    Google Scholar 

  31. Argonne National Lab. https://www.anl.gov. Last accessed 15 May 2023

  32. Phantom device. https://www.3dsystems.com/haptics-devices/3d-systems-phantom-premium. Last accessed 15 May 2023

  33. CyberGrasp. http://www.cyberglovesystems.com/cybergrasp. Last accessed 15 May 2023

  34. Pacchierotti C, Sinclair S, Solazzi M, Frisoli A, Hayward V et al (2017) Wearable haptic systems for the fingertip and the hand: taxonomy, review, and perspectives. IEEE Trans Haptics (ToH) 10(4):580–600

    Article  Google Scholar 

  35. Ultraleap. https://www.ultraleap.com/haptics/. Last accessed 15 May 2023

  36. Ni5vrglove. https://hi5vrglove.com. Last accessed 15 May 2023

  37. HaprX. https://haptx.com. Last accessed 15 May 2023

  38. Teslasuit. https://teslasuit.io. Last accessed 15 May 2023

  39. Ultrahaptics. https://www.ultraleap.com/haptics/. Last accessed 15 May 2023

  40. Ferrise F, Bordegoni M, Lizaranzu J (2010) Product design review application based on a vision-sound-haptic interface, haptic and audio interaction design conference. Lecture notes in computer science (LNCS), vol 6306/2010. Springer, pp 169–178

    Google Scholar 

  41. Ferrise F, Bordegoni M, Lizaranzu J (2011) Use of interactive virtual prototypes to define product design specifications: a pilot study on consumer product. In: Proceedings of IEEE-ISVRI, Singapore

    Google Scholar 

  42. Nakamoto T (2013) Human olfactory displays and interfaces: odor sensing and presentation. Inf Sci Ref

    Google Scholar 

  43. Porcherot C, Delplanque S, Raviot-Derrien S, Le Calvé B, Chrea C, Gaudreau N, Cayeux I (2010) How do you feel when you smell this? Optimization of a verbal measurement of odor-elicited emotions. Food Qual Prefer 21:938–947

    Article  Google Scholar 

  44. Rétiveau AN, Chambers E IV, Milliken GA (2004) Common and specific effects of fine fragrances on the mood of women. J Sens Stud 19:373–394

    Article  Google Scholar 

  45. Corbin A (1986) The foul and the fragrant: Odour and the French social imagination. Harvard University Press

    Google Scholar 

  46. Spangenberg ER, Crowley AE (1996) Improving the store environment: do olfactory cues affect evaluations and behaviors? J Mark 60(2):67–80

    Article  Google Scholar 

  47. Bosmans A (2006) Scents and sensibility: when do (in)congruent ambient scents influence product evaluations? J Mark 70(3):32–43

    Google Scholar 

  48. Gatti E, Bordegoni M, Spence C (2014) Investigating the influence of colour, weight, and fragrance intensity on the perception of liquid bath soap: an experimental study. Food Qual Prefer 31:56–64

    Article  Google Scholar 

  49. Demattè ML, Osterbauer R, Spence C (2007) Olfactory cues modulate facial attractiveness. Chem Senses 32(6):603–610

    Article  Google Scholar 

  50. Bradford KD, Desrochers DM (2009) The use of scents to influence consumers: the sense of using scents to make cents. J Bus Ethics 90:141–153

    Article  Google Scholar 

  51. Moessnang C, Finkelmeyer A, Vossen A, Schneider F, Habel U (2011) Assessing implicit odor localization in humans using a cross-modal spatial Cueing paradigm. PLoS One 6(12)

    Google Scholar 

  52. Gottfried JA, Dolan RJ (2003) The nose smells what the eye sees: Crossmodal visual facilitation of human olfactory perception. Neuron 39(2):375–386

    Article  Google Scholar 

  53. Zhou W, Zhang X, Chen J, Wang L, Chen D (2012) Nostril-specific olfactory modulation of visual perception in binocular rivalry. J Neurosci 32(48):17225–17229

    Article  Google Scholar 

  54. Blackwell L (1995) Visual cues and their effects on odour assessment. Nutr Food Sci 95(5):24–28

    Article  Google Scholar 

  55. Yanagida Y, Tomono A (2012) Basics for olfactory display, in human olfactory displays and interfaces. In: Nakamoto T (ed) Odor sensing and presentation. IGI Global

    Google Scholar 

  56. Vaqso. https://vaqso.com. Last accessed 15 May 2023

  57. Feelreal. https://feelreal.com/. Last accessed 15 May 2023

  58. Olorama. https://www.olorama.com/en/. Last accessed 15 May 2023

  59. Carulli M, Bordegoni M, Cugini U (2015) Integrating scents simulation in virtual reality multisensory environment for industrial products evaluation. Comput Aided Design Appl

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, Politecnico di Milano, Milan, Italy

    Monica Bordegoni & Marina Carulli

  2. Department of Design, Politecnico di Milano, Milan, Italy

    Elena Spadoni

Authors
  1. Monica Bordegoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marina Carulli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elena Spadoni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Monica Bordegoni .

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Bordegoni, M., Carulli, M., Spadoni, E. (2023). Multisensory Interaction in eXtended Reality. In: Prototy** User eXperience in eXtended Reality. SpringerBriefs in Applied Sciences and Technology(). Springer, Cham. https://doi.org/10.1007/978-3-031-39683-0_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-39683-0_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-39682-3

  • Online ISBN: 978-3-031-39683-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation