Abstract
The brain has the remarkable ability to bind together inputs from different sensory origin into a coherent percept. Behavioral benefits can result from such ability, e.g., a person typically responds faster and more accurately to cross-modal stimuli than to unimodal stimuli. To date, it is, however, largely unknown whether such multisensory benefits, shown for discrete reactive behaviors, generalize to the continuous coordination of movements. The present study addressed multisensory integration from the perspective of bimanual coordination dynamics, where the perceptual activity no longer triggers a single response but continuously guides the motor action. The task consisted in coordinating anti-symmetrically the continuous flexion–extension of the index fingers, while synchronizing with an external pacer. Three different configurations of metronome were tested, for which we examined whether a cross-modal pacing (audio–tactile beats) improved the stability of the coordination in comparison with unimodal pacing condition (auditory or tactile beats). We found a more stable bimanual coordination for cross-modal pacing, but only when the metronome configuration directly matched the anti-symmetric coordination pattern. We conclude that multisensory integration can benefit the continuous coordination of movements; however, this is constrained by whether the perceptual and motor activities match in space and time.
Similar content being viewed by others
References
Alais D, Newell FN, Mamassian P (2010) Multisensory processing in review: from physiology to behaviour. Seeing Perceiving 23:3–38
Aramaki Y, Honda M, Okada T, Sadato N (2006) Neural correlates of the spontaneous phase transition during bimanual coordination. Cereb Cortex 16(9):1338–1348
Assisi CG, Jirsa VK, Kelso JAS (2005) Dynamics of multifrequency coordination using parametric driving: theory and experiment. Biol Cybern 93(1):6–21
Barnejee A, Tognoli E, Kelso JAS, Jirsa V (2012) Spatiotemporal re-organization of large-scale neural assemblies underlies bimanual coordination. Neuroimage 62(3):1582–1592
Bell AH, Meredith MA, Van Opstal AJ, Munoz DP (2005) Crossmodal integration in the primate superior colliculus underlying the preparation and initiation of saccadic eye movements. J Neurophysiol 93(6):3659–3673
Bernatzky G, Bernatzky P, Hesse HP, Staffen W, Ladurner G (2004) Stimulating music increases motor coordination in patients afflicted with Morbus Parkinson. Neurosci Lett 361(1–3):4–8
Bloem BR, Hausdorff JM, Visser JE, Giladi N (2004) Falls and freezing of gait in Parkinson’s disease: a review of two interconnected, episodic phenomena. Mov Disord 19(8):871–884
Byblow WD, Carson RG, Goodman D (1994) Expressions of asymmetries and anchoring in bimanual coordination. Hum Mov Sci 13:3–28
Byblow WD, Summers JJ, Thomas J (2000) Spontaneous and intentional dynamics of bimanual coordination in Parkinson’s disease. Hum Mov Sci 19:223–249
Cappe C, Barone P (2005) Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. Eur J Neurosci 22:2886–2902
Cappe C, Morel A, Barone P, Rouiller EM (2009) The thalamocortical projection systems in primate: an anatomical support for multisensory and sensorimotor interplay. Cereb Cortex 19(9):2025–2037
Cassirer E (1944) The concept of group and the theory of perception. Philos Phenomenol Res 5(1):1–36
Diederich A, Colonius H (2004) Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept Psychophys 66(8):1388–1404
Diederich A, Schomburg A, Colonius H (2012) Saccadic reaction times to audiovisual stimuli show effects of oscillatory phase reset. PLoS One 7(10):e44910
Diedrichsen J, Wiestler T, Krakauer JW (2013) Two distinct ipsilateral cortical representations for individuated finger movements. Cereb Cortex 23(6):1362–1377
Driver J, Noesselt T (2008) Multisensory interplay reveals crossmodal influences on sensory-specific’s brain regions, neural responses, and judgments. Neuron 57(1):11–23
Elliott MT, Wing AM, Welchman AE (2010) Multisensory cues improve sensorimotor synchronisation. Eur J Neurosci 31(10):1828–1835
Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8:162–169
Fiebelkorn IC, Foxe JJ, Butler JS, Mercier MR, Snyder AC, Molholm S (2011) Ready, set, reset: stimulus-locked periodicity in behavioral performance demonstrates the consequences of cross-sensory phase reset. J Neurosci 31(27):9971–9981
Fink PW, Foo P, Jirsa VK, Kelso JA (2000) Local and global stabilization of coordination by sensory information. Exp Brain Res 134(1):9–20
Foxe JJ, Schroeder CE (2005) The case for feedforward multisensory convergence during early cortical processing. NeuroReport 16:419–423
Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE (2000) Multisensory auditory–somatosensory interactions in early cortical processing revealed by high-density electrical map**. Cogn Brain Res 10:77–83
Frens MA, Van Opstal AJ, Van der Willigen RF (1995) Spatial and temporal factors determine auditory–visual interactions in human saccadic eye movements. Attent Percept Psycho 57:802–816
Fu KMG, Johnston TA, Shah AS, Arnold L, Smiley J, Hackett TA, Garraghty PE, Schroeder CE (2003) Auditory cortical neurons respond to somatosensory stimulation. J Neurosci 23:7510–7515
Ghazanfar AA, Schroeder CE (2006) Is neocortex essentially multisensory? Trends Cogn Sci 10(6):278–285
Gillmeister H, Eimer M (2007) Tactile enhancement of auditory detection and perceived loudness. Brain Res 1160:58–68
Grefkes C, Eickhoff SB, Nowak DA, Dafotakis M, Fink GR (2008) Dynamic intra-and interhemispheric interactions during unilateral and bilateral hand movements assessed with fMRI and DCM. Neuroimage 41(4):1382–1394
Guest S, Catmur C, Lloyd D, Spence C (2002) Audiotactile interactions in roughness perception. Exp Brain Res 146:161–171
Hackett TA, De La Mothe LA, Ulbert I, Karmos G, Smiley J, Schroeder CE (2007a) Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane. J Comp Neuro 502(6):924–952
Hackett TA, Smiley JF, Ulbert I, Karmos G, Lakatos P, de la Mothe LA, Schroeder CE (2007b) Sources of somatosensory input to the caudal belt areas of auditory cortex. Perception 36(10):1419–1430
Harris CM, Wolpert DM (1998) Signal-dependent noise determines motor planning. Nature 394(6695):780–784
Hinder MR (2011) Interhemispheric connectivity between distinct motor regions as a window into bimanual coordination. J Neurophysiol 107(7):1791–1794
Holmes NP, Spence C (2005) Multisensory integration: space, time and superadditivity. Curr Biol 15(18):R762–R764
Hommel B (1996) SR compatibility effects without response uncertainty. Q J Exp Psychol A 49(3):546–571
Howard IS, Ingram JN, Körding KP, Wolpert DM (2009) Statistics of natural movements are reflected in motor errors. J Neurophysiol 102(3):1902–1910
Hughes HC, Reuter-Lorenz PA, Nozawa G, Fendrich R (1994) Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. J Exp Psychol Hum Percept Perform 20(1):131–153
Iacoboni M, Woods RP, Mazziotta JC (1998) Bimodal (auditory and visual) left frontoparietal circuitry for sensorimotor integration and sensorimotor learning. Brain 121(11):2135–2143
Jirsa VK, Ding M (2004) Will a large complex system with time delays be stable? Phys Rev Lett 93(7):070602
Jirsa VK, Fink P, Foo P, Kelso JAS (2000) Parametric stabilization of biological coordination: a theoretical model. J Biol Phys 26(2):85–112
Jousmäki V, Hari R (1998) Parchment-skin illusion: sound-biased touch. Curr Biol 8(6):R190
Kadunce DC, Vaughan JW, Wallace MT, Benedek G, Stein BE (1997) Mechanisms of within-and cross-modality suppression in the superior colliculus. J Neurophysiol 78:2834–2847
Kayser C, Logothetis NK (2007) Do early sensory cortices integrate cross-modal information? Brain Struct Funct 212(2):121–132
Kayser C, Petkov CI, Augath M, Logothetis NK (2005) Integration of touch and sound in auditory cortex. Neuron 48:373–384
Kelso JA (1984) Phase transitions and critical behavior in human bimanual coordination. Am J Physiol 246:R1000–R1004
Kelso JAS (1995) Dynamic patterns: the self-organization of brain and behavior. The MIT Press, Cambridge
Kelso JAS, Holt KG, Rubin P, Kugler PN (1981) Patterns of human interlimb coordination emerge from the properties of nonlinear limit cycle processes: theory and data. J Mot Behav 13(4):226–261
Kelso JAS, DelColle JD, Schöner G (1990) Action-perception as a pattern formation process. In: Jeannerod M (ed) Attention and performance, vol 13. Erlbaum, Hillsdale, pp 139–169
Lagarde J, Kelso JAS (2006) Binding of movement, sound and touch: multimodal coordination dynamics. Exp Brain Res 173(4):673–688
Lagarde J, Zelic G, Mottet D (2012) Segregated audio–tactile events destabilize the bimanual coordination of distinct rhythms. Exp Brain Res 219(3):409–419
Lakatos P, Chen CM, O’Connell MN, Mills A, Schroeder CE (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53(2):279–292
Lakatos P, Karmos G, Mehta AD, Ulbert I, Schroeder CE (2008) Entrainment of neuronal oscillations as a mechanism of attentional selection. Science 320(5872):110–113
Lewis GN, Byblow WD (2004) Bimanual coordination dynamics in poststroke hemiparetics. J Mot Behav 36(2):174–188
Liuzzi G, Hörnib V, Zimerman M, Gerloff C, Hummel FC (2011) Coordination of uncoupled bimanual movements by strictly timed interhemispheric connectivity. J Neurosci 31(25):9111–9117
Meister IG, Foltys H, Gallea C, Hallett M (2010) How the brain handles temporally uncoupled bimanual movements. Cereb Cortex 20(12):2996–3004
Meredith MA, Stein BE (1986) Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res 365(2):350–354
Meredith MA, Stein BE (1996) Spatial determinants of multisensory integration in cat superior colliculus neurons. J Neurophysiol 75:1843–1857
Meredith MA, Nemitz JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 7(10):3215–3229
Meyer-Lindenberg A, Ziemann U, Hajak G, Cohen L, Berman KF (2002) Transitions between dynamical states of differing stability in the human brain. Proc Natl Acad Sci USA 99(17):10948–10953
Miller J (1982) Divided attention: evidence for coactivation with redundant signals. Cogn Psychol 14(2):247–279
Miller LM, D’Esposito M (2005) Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J Neurosci 25:5884–5893
Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC, Schroeder CE, Foxe JJ (2005) Grabbing your ear: rapid auditory–somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15(7):963–974
Nourrit-Lucas D, Zelic G, Deschamps T, Hilpron M, Delignières D (2013) Persistent coordination patterns in a complex task after 10 years delay: subtitle: how validate the old saying “once you have learned how to ride a bicycle, you never forget!”. Hum Mov Sci 32(6):1365–1378
Perneger TV (1998) What’s wrong with Bonferroni adjustments. Br Med J 316(7139):1236–1238
Perrault TJ Jr, Vaughan JW, Stein BE, Wallace MT (2005) Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. J Neurophysiol 93:2575–2586
Pinto CM, Golubitsky M (2006) Central pattern generators for bipedal locomotion. J Math Biol 53(3):474–489
Raab DH (1962) Statistical facilitation of simple reaction times. Trans NY Acad Sci 24:574–590
Ronsse R, Miall RC, Swinnen SP (2009) Multisensory integration in dynamical behaviors: maximum likelihood estimation across bimanual skill learning. J Neurosci 29(26):8419–8428
Scheffer M, Bascompte J, Brock WA, Brovkin V, Carpenter SR, Dakos V, Sugihara G (2009) Early-warning signals for critical transitions. Nature 461(7260):53–59
Schmidt RA, Zelaznik HN, Frank JS (1978) Sources of inaccuracy in rapid movement. In: Stelmach GE (ed) Information processing in motor control and learning. Academic Press Inc., New York, pp 183–203
Schöner G, Kelso JAS (1988a) A synergetic theory of environmentally-specified and learned patterns of movement coordination. II. Component oscillator dynamics. Biol Cybern 58(2):81–89
Schöner G, Kelso JA (1988b) Dynamic pattern generation in behavioral and neural systems. Science 239(4847):1513–1520
Schöner G, Jiang WY, Kelso JS (1990) A synergetic theory of quadrupedal gaits and gait transitions. J Theor Biol 142(3):359–391
Schroeder CE, Foxe JJ (2004) Multisensory convergence in early cortical processing. In: Calvert GA (ed) The handbook of multisensory processes. MIT Press, Cambridge, pp 295–309
Schroeder CE, Foxe J (2005) Multisensory contributions to low-level, ‘unisensory’processing. Curr Opin Neurobiol 15(4):454–458
Schroeder CE, Lindsley RW, Specht C, Marcovici A, Smiley JF, Javitt DC (2001) Somatosensory input to auditory association cortex in the macaque monkey. J Neurophysiol 85:1322–1327
Schroeder CE, Smiley J, Fu KG, McGinnis T, O’Connell MN, Hackett TA (2003) Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int J Psychophysiol 50(1–2):5–17
Schürmann M, Caetano G, Jousmäki V, Hari R (2004) Hands help hearing: facilitatory audiotactile interaction at low sound-intensity levels. J Acoust Soc Am 115(2):830–832
Selen LP, Beek PJ, van Dieën JH (2005) Can co-activation reduce kinematic variability? A simulation study. Biol Cybern 93(5):373–381
Senkowski D, Schneider TR, Foxe JJ, Engel AK (2008) Crossmodal binding through neural coherence: implications for multisensory processing. Trends Neurosci 31(8):401–409
Smiley JF, Hackett TA, Ulbert I, Karmas G, Lakatos P, Javitt DC, Schroeder CE (2007) Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys. J Comp Neurol 502(6):894–923
Sperdin HF, Cappe C, Foxe JJ, Murray MM (2009) Early, low-level auditory–somatosensory multisensory interactions impact reaction time speed. Front Integr Neurosci 3:2
Sperdin HF, Cappe C, Murray MM (2010) Auditory–somatosensory multisensory interactions in humans: dissociating detection and spatial discrimination. Neuropsychologia 48(13):3696–3705
Stanford TR, Quessy S, Stein BE (2005) Evaluating the operations underlying multisensory integration in the cat superior colliculus. J Neurosci 25:6499–6508
Steenbergen B, Hulstijn W, de Vries A, Berger M (1996) Bimanual movement coordination in spastic hemiparesis. Exp Brain Res 110:91–98
Stein BE, Meredith MA (1993) The merging of the senses. The MIT Press, Cambridge
Stein BE, Stanford TR (2008) Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci 9(4):255–266
Stein BE, Stanford TR, Ramachandran R, Perrault TJ, Rowland BA (2009) Challenges in quantifying multisensory integration: alternative criteria, models, and inverse effectiveness. Exp Brain Res 198:113–126
Swinnen SP (2002) Intermanual coordination: from behavioural principles to neural-network interactions. Nat Rev Neurosci 3(5):348–359
Tagliabue M, McIntyre J (2011) Necessity is the mother of invention: reconstructing missing sensory information in multiple, concurrent reference frames for eye–hand coordination. J Neurosci 31(4):1397–1409
Thorne JD, De Vos M, Viola FC, Debener S (2011) Cross-modal phase reset predicts auditory task performance in humans. J Neurosci 31(10):3853–3861
Tuller B, Kelso JAS (1989) Environmentally-specified patterns of movement coordination in normal and split-brain subjects. Exp Brain Res 75:306–316
Turvey MT (1990) Coordination. Am Psychol 45(8):938–953
Turvey MT, Romaniak-Gross C, Isenhower RW, Arzamarski R, Harrison S, Carello C (2009) Human odometer is gait-symmetry specific. Proc Biol Sci 276(1677):4309–4314
Umiltà C, Nicoletti R (1990) Spatial stimulus-response compatibility. In: Proctor RW, Reeve TG (eds) Stimulus-response compatibility: an integrated perspective. North-Holland, Amsterdam, pp 89–116
Van Atteveldt N, Murray MM, Thut G, Schroeder CE (2014) Multisensory integration: flexible use of general operations. Neuron 81(6):1240–1253
Van Beers RJ, Haggard P, Wolpert DM (2004) The role of execution noise in movement variability. J Neurophysiol 91(2):1050–1063
Wallace MT, Wilkinson LK, Stein BE (1996) Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76:1246–1266
Zampini M, Torresan D, Spence C, Murray MM (2007) Auditory–somatosensory multisensory interactions in front and rear space. Neuropsychologia 45(8):1869–1877
Zanone PG, Kelso JA (1992) Evolution of behavioral attractors with learning: non equilibrium phase transitions. J Exp Psychol Hum Percept Perform 18(2):403–421
Zelic G, Mottet D, Lagarde J (2012) Behavioral impact of unisensory and multisensory audio–tactile events: pros and cons for interlimb coordination in juggling. PLoS One 7(2):e32308
Zelic G, Kim J, Davis C (2015) Articulatory constraints on spontaneous entrainment between speech and manual gesture. Hum Mov Sci 42:232–245
Acknowledgments
This research was supported by SKILLS, an Integrated Project (FP6-IST contract #035005) of the Commission of the European Community. We also gratefully acknowledge the input of the reviewers, with special thanks for the remarkable interaction about the race model predictions that definitely helped us to improve the paper from its original version.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing financial interests.
Rights and permissions
About this article
Cite this article
Zelic, G., Mottet, D. & Lagarde, J. Perceptuo-motor compatibility governs multisensory integration in bimanual coordination dynamics. Exp Brain Res 234, 463–474 (2016). https://doi.org/10.1007/s00221-015-4476-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-015-4476-5