Abstract
Information coming from multiple senses, as compared to a single one, typically enhances our performance. The multisensory improvement has been extensively examined in perception studies, as well as in tasks involving a motor response like a simple reaction time. However, how this effect extends to more complex behavior, typically involving the coordination of movements, such as bimanual coordination or walking, is still unclear. A critical element in achieving motor coordination in complex behavior is its stability. Reaching a stable state in the coordination pattern allows to sustain complex behavior over time (e.g., without interruption or negative consequences, like falling). This study focuses on the relation between stability in the coordination of movement patterns, like walking, and multisensory improvement. Participants walk with unimodal and audio-tactile metronomes presented either at their preferred rate or at a slower walking rate, the instruction being to synchronize their steps to the metronomes. Walking at a slower rate makes gait more variable than walking at the preferred rate. Interestingly however, the multimodal stimuli enhance the stability of motor coordination but only in the slower condition. Thus, the reduced stability of the coordination pattern (at a slower gait rate) prompts the sensorimotor system to capitalize on multimodal stimulation. These findings provide evidence of a new link between multisensory improvement and behavioral stability, in the context of ecological sensorimotor task.
Similar content being viewed by others
References
Albouy, P., Lévêque, Y., Hyde, K. L., Bouchet, P., Tillmann, B., & Caclin, A. (2015). Boosting pitch encoding with audiovisual interactions in congenital amusia. Neuropsychologia, 67, 111–120. https://doi.org/10.1016/j.neuropsychologia.2014.12.006.
Ammirante, P., Patel, A. D., & Russo, F. A. (2016). Synchronizing to auditory and tactile metronomes: a test of the auditory-motor enhancement hypothesis. Psychonomic Bulletin and Review. https://doi.org/10.3758/s13423-016-1067-9.
Armstrong, A., & Issartel, J. (2014). Sensorimotor synchronization with audio-visual stimuli: limited multisensory integration. Experimental Brain Research, 232(11), 3453–3463. https://doi.org/10.1007/s00221-014-4031-9.
Aschersleben, G., & Bertelson, P. (2003). Temporal ventriloquism: crossmodal interaction on the time dimension. 2. Evidence from sensorimotor synchronization. International Journal of Psychophysiology, 50, 157–163. https://doi.org/10.1016/S0167-8760.
Blais, M., Albaret, J.-M., & Tallet, J. (2015). Is there a link between sensorimotor coordination and inter-manual coordination? Differential effects of auditory and/or visual rhythmic stimulations. Experimental Brain Research, 233(11), 3261–3269. https://doi.org/10.1007/s00221-015-4394-6.
Blais, M., Martin, E., Albaret, J.-M., & Tallet, J. (2014). Preservation of perceptual integration improves temporal stability of bimanual coordination in the elderly: An evidence of age-related brain plasticity. Behavioural Brain Research, 275, 34–42. https://doi.org/10.1016/j.bbr.2014.08.043.
Bolognini, N., Convento, S., Rossetti, A., & Merabet, L. B. (2013). Multisensory processing after a brain damage: Clues on post-injury crossmodal plasticity from neuropsychology. Neuroscience and Biobehavioral Reviews, 37(3), 269–278. https://doi.org/10.1016/j.neubiorev.2012.12.006.
Bressler, S., & Kelso, J. (2001). Cortical coordination dynamics. Trends in Cognitive Sciences, 5(1), 26–36. https://doi.org/10.1016/S1364-6613(00)01564-3.
Bruijn, S. M., Meijer, O. G., Beek, P. J., & Van Dieën, J. H. (2013). Assessing the stability of human locomotion : a review of current measures. Journal of the Royal Society, Interface/the Royal Society. https://doi.org/10.1098/rsif.2012.0999.
Bruijn, S. M., van Dieën, J. H., Meijer, O. G., & Beek, P. J. (2009). Is slow walking more stable? Journal of Biomechanics, 42(10), 1506–1512. https://doi.org/10.1016/j.jbiomech.2009.03.047.
Caclin, A., Bouchet, P., Djoulah, F., Pirat, E., Pernier, J., & Giard, M. H. (2011). Auditory enhancement of visual perception at threshold depends on visual abilities. Brain Research, 1396, 35–44. https://doi.org/10.1016/j.brainres.2011.04.016.
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Hillsdale: Erlbaum Associates.
Colonius, H., & Diederich, A. (2004). Multisensory interaction in saccadic reaction time: a time-window-of-integration model. Journal of Cognitive Neuroscience, 16(6), 1000–1009. https://doi.org/10.1162/0898929041502733.
Dakos, V., Van Nes, E. H., D’Odorico, P., & Scheffer, M. (2012). Robustness of variance and autocorrelation as indicators of critical slowing down. Ecology, 93(2), 264–271.
Dalla Bella, S. (2018). Music and movement: Towards a translational approach. Neurophysiologie Clinique, 48(6), 377–386.
Dingwell, J., & Marin, L. (2006). Kinematic variability and local dynamic stability of upper body motions when walking at different speeds. Journal of Biomechanics, 39(3), 444–452. https://doi.org/10.1016/j.jbiomech.2004.12.014.
Elliott, M. T., Wing, A. M., & Welchman, A. E. (2010). Multisensory cues improve sensorimotor synchronisation. European Journal of Neuroscience, 31(10), 1828–1835. https://doi.org/10.1111/j.1460-9568.2010.07205.x.
Elliott, Mark T., Wing, A. M., & Welchman, A. E. (2011). The effect of ageing on multisensory integration for the control of movement timing. Experimental Brain Research, 213(2–3), 291–298. https://doi.org/10.1007/s00221-011-2740-x.
Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), 429–433. https://doi.org/10.1038/415429a.
Ernst, M. O., & Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences, 8(4), 162–169. https://doi.org/10.1016/j.tics.2004.02.002.
Frassinetti, F., Bolognini, N., Bottari, D., Bonora, A., & Làdavas, E. (2005). Audiovisual integration in patients with visual deficit. Journal of Cognitive Neuroscience, 17(9), 1442–1452. https://doi.org/10.1162/0898929054985446.
Gardiner, C. W. (2003). Handbook of stochastic methods for physics, chemistry and the natural sciences. Berlin: Springer.
Giard, M. H., & Peronnet, F. (1999). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience, 11(5), 473–490. https://doi.org/10.1162/089892999563544.
Haken, H., Kelso, J. S., & Bunz, H. (1985). A theoretical model of phase transitions in human hand movements. Biological Cybernetics, 51(5), 347–356.
Hershenson, M. (1962). Reaction time as a measure of intersensory facilitation. Journal of Experimental Psychology, 63(3), 289.
Huys, R., Perdikis, D., & Jirsa, V. K. (2014). Functional architectures and structured flows on manifolds: A dynamical framework for motor behavior. Psychological Review, 121(3), 302.
Jordan, K., Challis, J. H., & Newell, K. M. (2007). Walking speed influences on gait cycle variability. Gait and Posture, 26(1), 128–134. https://doi.org/10.1016/j.gaitpost.2006.08.010.
Kato, M., & Konishi, Y. (2006). Auditory dominance in the error correction process: A synchronized tap** study. Brain Research, 1084(1), 115–122. https://doi.org/10.1016/j.brainres.2006.02.019.
Kelso, J. A. S. (1995). Dynamic patterns: the self-organization of brain and behavior. Cambridge: MIT Press.
Kelso, J. A. S., Fink, P. W., DeLaplain, C. R., & Carson, R. G. (2001). Haptic information stabilizes and destabilizes coordination dynamics. Proceedings of the Royal Society B: Biological Sciences, 268(1472), 1207–1213. https://doi.org/10.1098/rspb.2001.1620.
Lagarde, J., & Kelso, J. A. S. (2006). Binding of movement, sound and touch: multimodal coordination dynamics. Experimental Brain Research, 173(4), 673–688. https://doi.org/10.1007/s00221-006-0410-1.
Lagarde, J., Zelic, G., & Mottet, D. (2012). Segregated audio-tactile events destabilize the bimanual coordination of distinct rhythms. Experimental Brain Research, 219(3), 409–419. https://doi.org/10.1007/s00221-012-3103-y.
Large, E. W., & Jones, M. R. (1999). The dynamics of attending: How people track time-varying events. Psychological Review, 106(1), 119.
Lovelace, C., Stein, B., & Wallace, M. (2003). An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Cognitive Brain Research, 17(2), 447–453.
Meredith, M., & Stein, B. (1986). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56(3), 640–662. https://doi.org/10.1152/jn.1986.56.3.640.
Meredith, M., Nemitz, J., & Stein, B. (1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. The Journal of Neuroscience, 7(10), 3215. https://doi.org/10.1523/JNEUROSCI.07-10-03215.1987.
Murray, M. M., Molholm, S., Michel, C. M., Heslenfeld, D. J., Ritter, W., Javitt, D. C., & Foxe, J. J. (2005). Grabbing your ear: Rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cerebral Cortex, 15(7), 963–974. https://doi.org/10.1093/cercor/bhh197.
Nelson, W., Hettinger, L., Cunningham, J., Brickman, B., Haas, M., & McKinley, R. (1998). Effects of localized auditory information on visual target detection performance using a helmet-mounted display. Human Factors, 40(3), 452–460.
Repp, B. H. (2005). Sensorimotor synchronization: A review of the tap** literature. Psychonomic Bulletin and Review, 12(6), 969–992. https://doi.org/10.3758/BF03206433.
Repp, B. H., & Penel, A. (2002). Auditory dominance in temporal processing: new evidence from synchronization with simultaneous visual and auditory sequences. Journal of Experimental Psychology Human Perception and Performance, 28(5), 1085–1099. https://doi.org/10.1037/0096-1523.28.5.1085.
Repp, B., & Penel, A. (2004). Rhythmic movement is attracted more strongly to auditory than to visual rhythms. Psychological Research Psychologische Forschung. https://doi.org/10.1007/s00426-003-0143-8.
Repp, B. H., & Su, Y.-H. (2013). Sensorimotor synchronization: A review of recent research (2006–2012). Psychonomic Bulletin and Review, 20(3), 403–452. https://doi.org/10.3758/s13423-012-0371-2.
Rowland, B. A., Quessy, S., Stanford, T. R., & Stein, B. E. (2007). Multisensory integration shortens physiological response latencies. Journal of Neuroscience, 27(22), 5879–5884. https://doi.org/10.1523/JNEUROSCI.4986-06.2007.
Roy, C., Dalla Bella, S., & Lagarde, J. (2017a). To bridge or not to bridge the multisensory time gap: bimanual coordination to sound and touch with temporal lags. Experimental Brain Research, 235(1), 135–151. https://doi.org/10.1007/s00221-016-4776-4.
Roy, C., Lagarde, J., Dotov, D., & Dalla Bella, S. (2017b). Walking to a multisensory beat. Brain and Cognition, 113, 172–183. https://doi.org/10.1016/j.bandc.2017.02.002.
Schöner, G., Haken, H., & Kelso, J. (1986). A stochastic theory of phase transitions in human hand movement. Biological Cybernetics, 53(4), 247–257. https://doi.org/10.1007/BF00336995.
Schöner, G., & Kelso, J. A. (1988). Dynamic pattern generation in behavioral and neural systems. Science, 239(4847), 1513–1520.
Sejdić, E., Fu, Y., Pak, A., Fairley, J. A., & Chau, T. (2012). The effects of rhythmic sensory cues on the temporal dynamics of human gait. PLoS One. https://doi.org/10.1371/journal.pone.0043104.
Sowiński, J., & Dalla Bella, S. (2013). Poor synchronization to the beat may result from deficient auditory-motor map**. Neuropsychologia, 51(10), 1952–1963. https://doi.org/10.1016/j.neuropsychologia.2013.06.027.
Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge: MIT Press.
Stein, B. E., & Stanford, T. R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nature Reviews Neuroscience, 9(4), 255–266. https://doi.org/10.1038/nrn2331.
Strogatz, S. H. (2018). Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering. Boca Raton: CRC Press.
Terrier, P., & Dériaz, O. (2012). Persistent and anti-persistent pattern in stride-to-stride variability of treadmill walking: influence of rhythmic auditory cueing. Human Movement Science, 31(6), 1585–1597. https://doi.org/10.1016/j.humov.2012.05.004.
Varlet, M., Marin, L., Issartel, J., Schmidt, R. C., & Bardy, B. G. (2012). Continuity of visual and auditory rhythms influences sensorimotor coordination. PLoS One. https://doi.org/10.1371/journal.pone.0044082.
Vroomen, J., & Keetels, M. (2010). Perception of intersensory synchrony: A tutorial review. Attention, Perception and Psychophysics, 72(4), 871–884. https://doi.org/10.3758/APP.72.4.871.
Warren, W. H. (2006). The dynamics of perception and action. Psychological Review, 113(2), 358.
Wright, R. L., & Elliott, M. T. (2014). Step** to phase-perturbed metronome cues: multisensory advantage in movement synchrony but not correction. Frontiers in Human Neuroscience, 8(September), 1–7. https://doi.org/10.3389/fnhum.2014.00724.
Yamasaki, M., Sasaki, T., Tsuzuki, S., & Torii, M. (1984). Stereotyped pattern of lower limb movement during level and grade walking on treadmill. The Annals of Physiological Anthropology, 3(4), 291–296. https://doi.org/10.1248/cpb.37.3229.
Zelic, Gregory, Mottet, D., & Lagarde, J. (2012). Behavioral impact of unisensory and multisensory audio-tactile events: Pros and cons for interlimb coordination in juggling. PLoS One, 7(2), e32308. https://doi.org/10.1371/journal.pone.0032308.
Zelic, G., Mottet, D., & Lagarde, J. (2016). Perceptuo-motor compatibility governs multisensory integration in bimanual coordination dynamics. Experimental Brain Research, 234(2), 463–474. https://doi.org/10.1007/s00221-015-4476-5.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors Charlotte Roy, Simone Dalla Bella, Simon Pla, and Julien Lagarde declare that they have not conflict of interest.
Ethical approval
The study was in accordance with the ethical standards of the national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Roy, C., Dalla Bella, S., Pla, S. et al. Multisensory integration and behavioral stability. Psychological Research 85, 879–886 (2021). https://doi.org/10.1007/s00426-019-01273-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00426-019-01273-4