Abstract
Motor control and motor learning as well as interpersonal coordination are based on motor perception and emergent perceptuomotor representations. At least in early stages motor learning and interpersonal coordination are emerging heavily on visual information in terms of observing others and transforming the information into internal representations to guide owns behavior. With progressing learning, also other perceptual modalities are added when a new motor pattern is established by repeated physical exercises. In contrast to the vast majority of publications on motor learning and interpersonal coordination referring to a certain perceptual modality here we look at the perceptual system as a unitary system coordinating and unifying the information of all involved perceptual modalities. The relation between perceptual streams of different modalities, the intermodal processing and multisensory integration of information as a basis for motor control and learning will be the main focus of this contribution.
Multi-/intermodal processing of perceptual streams results in multimodal representations and opens up new approaches to support motor learning and interpersonal coordination: Creating an additional perceptual stream adequately auditory movement information can be generated suitable to be integrated with information of other modalities and thereby modulating the resulting perceptuomotor representations without the need of attention and higher cognition. Here, the concept of a movement defined real-time acoustics is used to serve the auditory system in terms of an additional movement-auditory stream. Before the computational approach of kinematic real-time sonification is finally described, a special focus is directed to the level of adaptation modules of the internal models. Furthermore, this concept is compared with different approaches of additional acoustic movement information. Moreover, a perspective of this approach is given in a broad spectrum of new applications of supporting motor control and learning in sports and motor rehabilitation as well as a broad spectrum of joint action and interpersonal coordination between humans but also concerning human-robot-interaction.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Asanuma, H., Keller, A.: Neuronal mechanisms of motor learning in mammals. NeuroReport 2(5), 217–224 (1991)
Atallah, L., Aziz, O., Lo, B., Yang, G.Z.: Detecting walking gait impairment with an ear-worn sensor. In: Sixth International Workshop on Wearable and Implantable Body Sensor Networks, BSN 2009, pp. 175–180. IEEE (2009)
Bastian, A.J.: Understanding sensorimotor adaptation and learning for rehabilitation. Curr. Opin. Neurol. 21(6), 628–633 (2008)
Baumann, O., Greenlee, M.W.: Neural correlates of coherent audiovisual motion perception. Cereb. Cortex 17, 1433–1443 (2006)
Bernier, P., Gauthier, G., Blouin, J.: Evidence for distinct, differentially adaptable sensorimotor transformations for reaches to visual and proprioceptive targets. J. Neurophysiol. 98, 1815–1819 (2007)
Berthoz, A., Viaud-Delmon, I.: Multisensory integration in spatial orientation. Curr. Opin. Neurobiol. 9(6), 708–712 (1999)
Bialkowski, S.E.: Real-time digital filters: infinite impulse response filters. Anal. Chem. 60(6), 403A–413A (1988)
Bishop, L., Goebl, W.: Beating time: how ensemble musicians’ cueing gestures communicate beat position and tempo. Psychol. Music 46(1), 84–106 (2018). https://doi.org/10.1177/0305735617702971
Bock, O.: Basic principles of sensorimotor adaptation to different distortions with different effectors and movement types: a review and synthesis of behavioral findings. Front. Hum. Neurosci. 7, 81 (2013)
Bock, O., Schmitz, G.: Transfer of visuomotor adaptation to unpractised hands and sensory modalities. Psychology 4(12), 1004–1007 (2013). https://doi.org/10.4236/psych.2013.412145
Bock, O., Schmitz, G., Grigorova, V.: Transfer of adaptation between ocular saccades and arm movements. Hum. Mov. Sci. 27, 383–395 (2008)
Brodie, M., Walmsley, A., Page, W.: Fusion motion capture: a prototype system using inertial measurement units and GPS for the biomechanical analysis of ski racing. Sports Technol. 1(1), 17–28 (2008)
Buchecker, M., Wegenkittl, S., Stöggl, T., Müller, E.: Unstable footwear affects magnitude and structure of variability in postural control. Motor Control 22(1), 1–35 (2017)
Calvert, G.A., Spence, C., Stein, B.E. (eds.): The Handbook of Multisensory Processes. MIT Press, Cambridge (2004)
Campos-Sousa, I.S., Campos-Sousa, R.N., Ataide Jr., L., Soares, M.M., Almeida, K.J.: Executive dysfunction and motor symptoms in Parkinson’s disease. Arq. Neuropsiquiatr. 68(2), 246–251 (2010)
Chen, L., Vroomen, Y.: Intersensory binding across space and time: a tutorial review. Atten. Percept. Psychophys. 75, 790–811 (2013)
Cohen, M.: Changes in auditory localization following prismatic exposure under continuous and terminal visual feedback. Percept. Mot. Skills 38, 1202 (1974)
Craske, B.: Intermodal transfer of adaptation to displacement. Nature 5037, 765 (1966)
Cromwell, R., Wellmon, R.: Sagittal plane head stabilization during level walking and ambulation on stairs. Physiotherapy Res. Int. 6(3), 179–192 (2001)
Cunnington, R., Iansek, R., Bradshaw, J.L., Phillips, J.G.: Movement-related potentials in Parkinson’s disease. Brain 118(4), 935–950 (1995)
D’Ausilio, A., Badino, L., Li, Y., Tokay, S., Craighero, L., Canto, R., Aloimonos, Y., Fadiga, L.: Leadership in orchestra emerges from the causal relationships of movement kinematics. PLoS ONE 7(5), e35757 (2012)
D’Ausilio, A., Novembre, G., Fadiga, L., Keller, P.E.: What can music tell us about social interaction? Trends Cogn. Sci. 19(3), 111–114 (2015)
Debaere, F., Wenderoth, N., Sunaert, S., Van Hecke, P., Swinnen, S.P.: Internal vs external generation of movements: differential neural pathways involved in bimanual coordination performed in the presence or absence of augmented visual feedback. Neuroimage 19(3), 764–776 (2003)
Delignières, D., Torre, K.: Fractal dynamics of human gait: a reassessment of the 1996 data of Hausdorff et al. J. Appl. Physiol. 106(4), 1272–1279 (2009)
Demos, A.P., Chaffin, R., Begosh, K.T., Daniels, J.R., Marsh, K.L.: Rocking to the beat: effects of music and partner’s movements on spontaneous interpersonal coordination. J. Exp. Psychol. Gen. 141(1), 49 (2012)
Dijkerman, H.C., McIntosh, R.D., Anema, H.A., de Haan, E.H., Kappelle, L.J., Milner, A.D.: Reaching errors in optic ataxia are linked to eye position rather than head or body position. Neuropsychologia 44(13), 2766–2773 (2006)
Dubus, G., Bresin, R.: A systematic review of map** strategies for the sonification of physical quantities. PLoS ONE 8(12), e82491 (2013)
Effenberg, A.O.: Movement sonification: effects on perception and action. IEEE Multimedia 12(2), 53–59 (2005)
Effenberg, A.O., Fehse, U., Schmitz, G., Krueger, B., Mechling, H.: Movement sonification: effects on motor learning beyond rhythmic adjustments. Front. Neurosci. 10 (2016). https://doi.org/10.3389/fnins.2016.00219
Effenberg, A.O., Schmitz, G., Baumann, F., Rosenhahn, B., Kroeger, D.: Soundscript–supporting the acquisition of character writing by multisensory integration. Open Psychol. J. 8(3), 230–237 (2015). https://doi.org/10.2174/1874350101508010230
Effenberg, A.O., Schmitz, G.: Acceleration and deceleration at constant speed: systematic modulation of motion perception by kinematic sonification. Ann. N. Y. Acad. Sci. (2018). https://doi.org/10.1111/nyas.13693
El-Tamawy, M.S., Darwish, M.H., Khallaf, M.E.: Effects of augmented proprioceptive cues on the parameters of gait of individuals with Parkinson’s disease. Ann. Indian Acad. Neurol. 15(4), 267–272 (2012)
Felix, R.A., Fridberger, A., Leijon, S., Berrebi, A.S., Magnusson, A.K.: Sound rhythms are encoded by postinhibitory rebound spiking in the superior paraolivary nucleus. J. Neurosci. 31(35), 12566–12578 (2011)
Flannigan, J.C., Posthuma, R.J., Lombardo, J.N., Murray, C., Cressmann, E.K.: Adaptation to proprioceptive targets following visuomotor adaptation. Exp. Brain Res. 236, 419–432 (2018)
Ford, M.P., Malone, L.A., Nyikos, I., Yelisetty, R., Bickel, C.S.: Gait training with progressive external auditory cueing in persons with Parkinson’s disease. Arch. Phys. Med. Rehabil. 91(8), 1255–1261 (2010)
Fujioka, T., Trainor, L.J., Large, E.W., Ross, B.: Internalized timing of isochronous sounds is represented in neuromagnetic beta oscillations. J. Neurosci. 32(5), 1791–1802 (2012)
Galea, J., Miall, R.: Concurrent adaptation to opposing visual displacements during an alternating movement. Exp. Brain Res. 175, 676–688 (2006)
Ghai, S., Ghai, I., Effenberg, A.O.: Effect of rhythmic auditory cueing on aging gait: a systematic review and meta-analysis. Aging Dis. 131–200 (2017a)
Ghai, S., Ghai, I., Effenberg, A.O.: Effects of dual-task training and dual-tasks on postural stability: a systematic review and meta-analysis. Clin. Interv. Aging 12, 557–577 (2017b)
Ghai, S., Ghai, I., Effenberg, A.O.: Effect of rhythmic auditory cueing on gait in cerebral palsy: a systematic review and meta-analysis. Neuropsychiatric Dis. Treat. 14, 43–59 (2018a)
Ghai, S., Ghai, I., Schmitz, G., Effenberg, A.O.: Effect of rhythmic auditory cueing on Parkinsonian gait: a systematic review and meta-analysis. Sci. Rep. 8(1), 506 (2018b)
Ghai, S., Schmitz, G., Hwang, T.-H., Effenberg, A.O.: Auditory proprioceptive integration: effects of real-time kinematic auditory feedback on knee proprioception. Front. Neurosci. 12, 142 (2018c)
Gibson, J.J.: The Senses Considered as Perceptual Systems. Houghton-Mifflin, Boston (1966)
Gibson, J.J.: The Ecological Approach to Visual Perception. Houghton-Mifflin, Boston (1979)
Goodman, J.R., Isenhower, R.W., Marsh, K., Schmidt, R., Richardson, M.: The interpersonal phase entrainment of rocking chair movements. In: Heft, H., Marsh, K.L. (eds.) Studies in Perception and Action VIII: Thirteenth International Conference on Perception and Action (2005)
Haar, S., Donchin, O., Dinstein, I.: Dissociating visual and motor directional selectivity using visuomotor adaptation. J. Neurosci. 35(17), 6813–6821 (2015)
Hameed, S., Ferris, T., Jayaraman, S., Sarter, N.: Using informative peripheral visual and tactile cues to support task and interruption management. Hum. Factors 51(2), 126–135 (2009)
Harris, C.S.: Adaptation to displaced vision: visual, motor, or proprioceptive change? Science 140, 812–813 (1963)
Harris, C.S.: Perceptual adaptation to inverted, reversed, and displaced vision. Psychol. Rev. 72(6), 419–444 (1965)
Hatada, Y., Miall, R.C., Rossetti, Y.: Two waves of a long-lasting aftereffect of prism adaptation measured over 7 days. Exp. Brain Res. 169(3), 417–426 (2006)
Hausdorff, J.M., Lowenthal, J., Herman, T., Gruendlinger, L., Peretz, C., Giladi, N.: Rhythmic auditory stimulation modulates gait variability in Parkinson’s disease. Eur. J. Neurosci. 26(8), 2369–2375 (2007)
Hausdorff, J.M., Purdon, P.L., Peng, C., Ladin, Z., Wei, J.Y., Goldberger, A.L.: Fractal dynamics of human gait: stability of long-range correlations in stride interval fluctuations. J. Appl. Physiol. 80(5), 1448–1457 (1996)
Hellmers, H., Norrdine, A., Blankenbach, J., Eichhorn, A.: An IMU/magnetometer-based indoor positioning system using Kalman filtering. In: 2013 International Conference on Indoor Positioning and Indoor Navigation (IPIN), pp. 1–9. IEEE (2013)
Hesch, J.A., Kottas, D.G., Bowman, S.L., Roumeliotis, S.I.: Camera-IMU-based localization: observability analysis and consistency improvement. Int. J. Robot. Res. 33(1), 182–201 (2014)
Hopkins, K., Kass, S.J., Blalock, L.D., Brill, J.C.: Effectiveness of auditory and tactile crossmodal cues in a dual-task visual and auditory scenario. Ergonomics 60(5), 692–700 (2017)
Hove, M.J., Suzuki, K., Uchitomi, H., Orimo, S., Miyake, Y.: Interactive rhythmic auditory stimulation reinstates natural 1/f timing in gait of Parkinson’s patients. PLoS ONE 7(3), e32600 (2012)
Hwang, T.H., Reh, J., Effenberg, A.O., Blume, H.: Real-time gait analysis using a single head-worn inertial measurement unit. IEEE Trans. Consum. Electron. 64(2), 240–248 (2018a). https://doi.org/10.1109/tce.2018.2843289
Hwang, T.-H., et al.: Effect and performance-based auditory feedback on interpersonal coordination. Front. Psychol. 9, 404 (2018b). https://doi.org/10.3389/fpsyg.2018.00404
Imamizu, H., Shimojo, S.: The locus of visual-motor learning at the task or manipulator level: implications from intermanual transfer. J. Exp. Psychol. Hum. Percept. Perform. 21, 719–733 (1995)
Imamizu, H., et al.: Explicit contextual information selectively contributes to predictive switching of internal models. Exp. Brain Res. 181(3), 395–408 (2007)
Johansson, B.B.: Multisensory stimulation in stroke rehabilitation. Front. Hum. Neurosci. 6, 60 (2012)
Kagerer, F.A., Contreras-Vidal, J.L.: Adaptation of sound localization induced by rotated visual feedback in reaching movements. Exp. Brain Res. 193(2), 315–321 (2009)
Keller, P.E., Knoblich, G., Repp, B.H.: Pianists duet better when they play with themselves: on the possible role of action simulation in synchronization. Conscious. Cogn. 16(1), 102–111 (2007)
Keller, P.E., Novembre, G., Hove, M.J.: Rhythm in joint action: psychological and neurophysiological mechanisms for real-time interpersonal coordination. Philos. Trans. Roy. Soc. B: Biol. Sci. 369(1658), 20130394 (2014)
Keysers, C., Kohler, E., Umilta, M.A., Nanetti, L., Fogassi, L., Gallese, V.: Audiovisual mirror neurons and action recognition. Exp. Brain Res. 153, 628–636 (2003)
Khoramshahi, M., Shukla, A., Raffard, S., Bardy, B.G., Billard, A.: Role of gaze cues in interpersonal motor coordination: towards higher affiliation in human-robot interaction. PLoS ONE 11(6), e0156874 (2016). https://doi.org/10.1371/journal.pone.0156874
Kirk, A.G., O’Brien, J.F., Forsyth, D.A.: Skeletal parameter estimation from optical motion capture data. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 782–788. IEEE (2005)
Knoblich, G., Butterfill, S., Sebanz, N.: Psychological research on joint action: theory and data. In: Psychology of Learning and Motivation-Advances in Research and Theory, vol. 54, p. 59 (2011)
Kohler, E., Keysers, C., Umilta, M.A., Fogassi, L., Gallese, V., Rizzolati, G.: Hearing sounds, understanding actions: action representation in mirror neurons. Science 297, 846–848 (2002)
Kohler, I.: Über Aufbau und Wandlungen der Wahrnehmungswelt, insbesondere über bedingte Empfindungen.’: In: Kommission bei RM Rohrer (1951)
Kohler, I.: The formation and transformation of the perceptual world. Psychological Issues (1963)
Lacquaniti, F., et al.: Multisensory integration and internal models for sensing gravity effects in primates. In: BioMed Research International (2014)
Lahav, A., Saltzman, E., Schlaug, G.: Action representation of sound: audiomotor recognition network while listening to newly acquired actions. J. Neurosci. 27(2), 308–314 (2007)
Lee, J.Y., Schweighofer, N.: Dual adaptation supports a parallel architecture of motor memory. J. Neurosci. 29(33), 10396–10404 (2009)
Lewald, J.M., Getzmann, S.: Horizontal and vertical effects of eye-position on sound localization. Hear. Res. 213, 99–106 (2006)
Lohnes, C.A., Earhart, G.M.: The impact of attentional, auditory, and combined cues on walking during single and cognitive dual tasks in Parkinson disease. Gait Posture 33(3), 478–483 (2011)
Magescas, F., Prablanc, C.: Automatic drive of limb motor plasticity. J. Cogn. Neurosci. 18(1), 75–83 (2006)
Magill, R.A., Anderson, D.I.: Motor Learning and Control: Concepts and Applications, vol. 11. McGraw-Hill, New York (2007)
Martin, L.M., Newman, C.V.: Simultaneous right- and left-hand adaptation in opposite lateral directions following bidirectional optical displacement. Bull. Psychon. Soc. 16(6), 432–434 (1980)
Michel, C., Pisella, L., Prablanc, C., Rode, G., Rossetti, Y.: Enhancing visuomotor adaptation by reducing error signals: single-step (Aware) versus multiplestep (Unaware) exposure to wedge prisms. J. Cogn. Neurosci. 19(2), 341–350 (2007)
Mikaelian, H.: Lack of bilateral generalization of adaptation to auditory rearrangement. Percept. Psychophys. 11(3), 222–224 (1972)
Mikaelian, H.: Adaptation to displaced hearing: a nonproprioceptive change. J. Exp. Psychol. 103, 326–330 (1974)
Miyata, K., Varlet, M., Miura, A., Kudo, K., Keller, P.E.: Modulation of individual auditory-motor coordination dynamics through interpersonal visual coupling. Sci. Rep. 7, 16220 (2017). https://doi.org/10.1038/s41598-017-16151-5
Moeslund, T.B., Hilton, A., Krueger, V.: A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104(2–3), 90–126 (2006)
Morton, S.M., Bastian, A.J.: Prism adaptation during walking generalizes to reaching and requires the cerebellum. J. Neurosci. 92, 2497–2509 (2004)
Mueller, G., Moeser, M. (eds.): Handbook of Engineering Acoustics. Springer Science and Business Media, Berlin (2012). https://doi.org/10.1007/978-3-540-69460-1
Murgia, M., et al.: Ecological sounds affect breath duration more than artificial sounds. Psychol. Res. 80(1), 76–81 (2016)
Nieuwboer, A., et al.: The short-term effects of different cueing modalities on turn speed in people with Parkinson’s disease. Neurorehabil. Neural Repair 23(8), 831–836 (2009)
Nieuwboer, A., et al.: Cueing training in the home improves gait-related mobility in Parkinson’s disease: the RESCUE trial. J. Neurol. Neurosurg. Psychiatry 78(2), 134–140 (2007)
Nombela, C., Hughes, L.E., Owen, A.M., Grahn, J.A.: Into the groove: can rhythm influence Parkinson’s disease? Neurosci. Biobehav. Rev. 37(10), 2564–2570 (2013)
Nunes, M.E., Souza, M.G., Basso, L., Monteiro, C., Corrêa, U.C., Santos, S.: Frequency of provision of knowledge of performance on skill acquisition in older persons. Front. Psychol. 5, 1454 (2014)
Oostwoud Wijdenes, L., Medendorp, W.P.: State estimation for early feedback responses in reaching: intramodal or multimodal? Front. Integr. Neurosci. 11, 38 (2017). https://doi.org/10.3389/fnint.2017.00038
Oppenheim, A.V.: Discrete-Time Signal Processing. Pearson Education India, Bangalore (1999)
Oscari, F., Secoli, R., Avanzini, F., Rosati, G., Reinkensmeyer, D.J.: Substituting auditory for visual feedback to adapt to altered dynamic and kinematic environments during reaching. Exp. Brain Res. 221, 33–41 (2012)
Oullier, O., De Guzman, G.C., Jantzen, K.J., Lagarde, J., Scott Kelso, J.: Social coordination dynamics: measuring human bonding. Soc. Neurosci. 3(2), 178–192 (2008)
Prablanc, C., Tzavaras, A., Jeannerod, M.: Adaptation of the two arms to opposite prism displacements. Q. J. Exp. Psychol. 27(4), 667–671 (1975)
Rinehart, N.J., Bellgrove, M.A., Tonge, B.J., Brereton, A.V., Howells-Rankin, D., Bradshaw, J.L.: An examination of movement kinematics in young people with high-functioning autism and Asperger’s disorder: further evidence for a motor planning deficit. J. Autism Dev. Disord. 36(6), 757–767 (2006)
Reh, J., Hwang, T.H., Michalke, V., Effenberg, A.O.: Instruction and real-time sonification for gait rehabilitation after unilateral hip arthroplasty. In: 11th Joint Conference on Motor Control Learning Biomechanics Training, pp. 1–2. DVS (2016)
Rios, J.A., White, E.: Fusion filter algorithm enhancements for a MEMS GPS/IMU, pp. 1–12. Crossbow Technology, Inc. (2002)
Rueterbories, J., Spaich, E.G., Larsen, B., Andersen, O.K.: Methods for gait event detection and analysis in ambulatory systems. Med. Eng. Phys. 32(6), 545–552 (2010)
Sainburg, R., Wang, J.: Interlimb transfer of visuomotor rotations: independence of direction and final position information. Exp. Brain Res. 145, 437–447 (2002)
Schaefer, R.S.: Auditory rhythmic cueing in movement rehabilitation: findings and possible mechanisms. Philos. Trans. Roy. Soc. B: Biol. Sci. 369, 20130402 (2014)
Schmitz, G., Bock, O.: Properties of intermodal transfer after dual visuo- and auditory-motor adaptation. Hum. Mov. Sci. 55, 108–120 (2017)
Schmitz, G.: Visuo- und Audiomotorische Adaptation. Hofmann-Verlag, Schorndorf (2014). ISBN 978-3-7780-4850-4
Schmitz, G.: Interference between adaptation to double steps and adaptation to rotated feedback in spite of differences in directional selectivity. Exp. Brain Res. 234, 1491–1504 (2016). https://doi.org/10.1007/s00221-016-4559-y
Schmitz, G., Effenberg, A.O.: Sound joint actions in rowing and swimming. In: Meyer, C., Wedelstaedt, U.V. (eds.) Moving Bodies in Interaction - Interacting Bodies in Motion. John Benjamins Publishing Company, Amsterdam (2016)
Schmitz, G., Bergmann, J., Effenberg, A.O., Krewer, C., Hwang, T.H., Mueller, F.: Movement sonification in stroke rehabilition. Front. Neurol. 9, 389 (2018)
Schmitz, G., Bock, O.: A comparison of sensorimotor adaptation in the visual and in the auditory modality. PLoS ONE 9(9), e107834 (2014)
Schmitz, G., Effenberg, A.O.: Perceptual effects of auditory information about own and other movements. In: 18th International Conference on Auditory Display, Atlanta, GA, USA (2012)
Schmitz, G., Effenberg, A.O.: Schlagmann 2.0 – Bewegungsakustische Dimensionen interpersonaler Koordination im Mannschaftssport. Ger. J. Exerc. Sport Res. 47(3), 232–245 (2017)
Schmitz, G., Grigorova, V.: Alternating adaptation of eye and hand movements to opposite directed double steps. J. Mot. Behav. 49(3), 255–264 (2017). https://doi.org/10.1080/00222895.2016.1191419
Schmitz, G., Kroeger, D., Effenberg, A.O.: A mobile sonification system for stroke rehabilitation. In: The 20th International Conference on Auditory Display, New York (2014)
Schmitz, G., et al.: Observation of sonified movements engages a basal ganglia frontocortical network. BMC Neurosci. 14, 32 (2013). https://doi.org/10.1186/1471-2202-14-32
Sebanz, N., Knoblich, G.: Prediction in joint action: what, when, and where. Top. Cogn. Sci. 1(2), 353–367 (2009)
Sebanz, N., Bekkering, H., Knoblich, G.: Joint action: bodies and minds moving together. Trends Cogn. Sci. 10(2), 70–76 (2006)
Seitz, A.R., Kim, R., Shams, L.: Sound facilitates visual learning. Curr. Biol. 16(14), 1422–1427 (2006)
Sengpielaudio homepage: http://www.sengpielaudio.com/calculator-notenames.htm
Shadmehr, R., Smith, R.A., Krakauer, J.W.: Error correction, sensory prediction, and adaptation in motor control. Ann. Rev. Neurosci. 33, 89–108 (2010)
Shams, L., Seitz, A.R.: Benefits of multisensory learning. Trends Cogn. Sci. 12(11), 411–417 (2008)
Sharma, D.A., Chevidikunnan, M.F., Khan, F.R., Gaowgzeh, R.A.: Effectiveness of knowledge of result and knowledge of performance in the learning of a skilled motor activity by healthy young adults. J. Phys. Ther. Sci. 28(5), 1482–1486 (2016)
Silaghi, M.-C., Plänkers, R., Boulic, R., Fua, P., Thalmann, D.: Local and global skeleton fitting techniques for optical motion capture. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) CAPTECH 1998. LNCS (LNAI), vol. 1537, pp. 26–40. Springer, Heidelberg (1998). https://doi.org/10.1007/3-540-49384-0_3
Smith, M.A., Ghazizadeh, A., Shadmehr, R.: Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 4(6), e179 (2006). https://doi.org/10.1371/journal.pbio.0040179
Spence, C.: Cross-modal perceptual organization. In: Wagemans, J. (ed.) The Oxford Handbook of Perceptual Organization. Oxford University Press, Oxford (2015)
Spence, C., Driver, J. (eds.): Crossmodal Space and Crossmodal Attention. Oxford University Press, Oxford (2004)
Stein, B.E., Meredith, M.A.: The Merging of the Senses. MIT Press, Cambridge (1993)
Stein, B.E., Stanford, T.R.: Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9(4), 255–266 (2008)
Stoffregen, T.A., Bardy, B.G.: On specification and the senses. Behav. Brain Sci. 24, 195–213 (2001). Discussion 213-161
Suh, J.H., et al.: Effect of rhythmic auditory stimulation on gait and balance in hemiplegic stroke patients. NeuroRehabilitation 34(1), 193–199 (2014)
Tagliabue, M., McIntyre, J.: A modular theory of multisensory integration for motor control. Front. Comput. Neurosci. 8, 1 (2014). https://doi.org/10.3389/fncom.2014.00001
Tecchio, F., Salustri, C., Thaut, M.H., Pasqualetti, P., Rossini, P.: Conscious and preconscious adaptation to rhythmic auditory stimuli: a magnetoencephalographic study of human brain responses. Exp. Brain Res. 135(2), 222–230 (2000)
Thaut, M.H.: Neural basis of rhythmic timing networks in the human brain. Ann. N. Y. Acad. Sci. 999(1), 364–373 (2003)
Thaut, M.H.: Rhythm, Music, and the Brain: Scientific Foundations and Clinical Applications, vol. 7. Routledge, Abingdon (2005)
Thaut, M.H., et al.: Neurologic music therapy improves executive function and emotional adjustment in traumatic brain injury rehabilitation. Ann. N. Y. Acad. Sci. 1169(1), 406–416 (2009)
Thaut, M.H., Leins, A.K., Rice, R.R., Argstatter, H., Kenyon, G.P., McIntosh, G.C., Fetter, M., et al.: Rhythmic auditory stimulation improves gait more than NDT/Bobath training in near-ambulatory patients early poststroke: a single-blind, randomized trial. Neurorehabil. Neural Repair 21(5), 455–459 (2007)
Thaut, M., Schleiffers, S., Davis, W.: Analysis of EMG activity in biceps and triceps muscle in an upper extremity gross motor task under the influence of auditory rhythm. J. Music Ther. 28(2), 64–88 (1991)
Tierney, A., Kraus, N.: The ability to move to a beat is linked to the consistency of neural responses to sound. J. Neurosci. 33(38), 14981–14988 (2013)
Torres, E.B., Heilman, K.M., Poizner, H.: Impaired endogenously evoked automated reaching in Parkinson’s disease. J. Neurosci. 31(49), 17848–17863 (2011)
Vesper, C., Butterfill, S., Knoblich, G., Sebanz, N.: A minimal architecture for joint action. Neural Netw. 23(8), 998–1003 (2010)
Weeks, D.L., Kordus, R.N.: Relative frequency of knowledge of performance and motor skill learning. Res. Q. Exerc. Sport 69(3), 224–230 (1998)
Welch, R.B.: Perceptual Modification. Adapting to Altered Sensory Environments. Academic Press, Cambridge (1978)
Whitall, J., et al.: Bilateral and unilateral arm training improve motor function through differing neuroplastic mechanisms a single-blinded randomized controlled trial. Neurorehabil. Neural Repair 25(2), 118–129 (2011)
Wigmore, V., Tong, C., Flanagan, J.R.: Visuomotor rotations of varying size and direction compete for single internal model in working memory. J. Exp. Psychol. Hum. Percept. Perform. 28, 447–457 (2002)
Winstein, C.J.: Knowledge of results and motor learning—implications for physical therapy. Phys. Ther. 71(2), 140–149 (1991)
Wolpert, D.M., Kawato, M.: Multiple paired forward and inverse models for motor control. Neural Netw. 11(7–8), 1317–1329 (1998)
Wolpert, D.M., Diedrichsen, J., Flanagan, J.R.: Principles of sensorimotor learning. Nat. Rev. Neurosci. 12, 739–751 (2011)
Wolpert, D.M., Ghahramani, Z., Jordan, M.I.: An internal model for sensorimotor integration. Science 269, 1880–1882 (1995)
Young, W.R., Rodger, M.W., Craig, C.M.: Auditory observation of step** actions can cue both spatial and temporal components of gait in Parkinson’s disease patients. Neuropsychologia 57, 140–153 (2014)
Zmigrod, S., Hommel, B.: Feature Integration across multimodal perception and action: a review. Multisensory Res. 26, 143–157 (2013)
Acknowledgments
This research was supported by European Commission H2020-FETPROACT-2014 No. 641321.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Effenberg, A.O., Hwang, TH., Ghai, S., Schmitz, G. (2018). Auditory Modulation of Multisensory Representations. In: Aramaki, M., Davies , M., Kronland-Martinet, R., Ystad, S. (eds) Music Technology with Swing. CMMR 2017. Lecture Notes in Computer Science(), vol 11265. Springer, Cham. https://doi.org/10.1007/978-3-030-01692-0_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-01692-0_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01691-3
Online ISBN: 978-3-030-01692-0
eBook Packages: Computer ScienceComputer Science (R0)