Abstract
Research results indicate that the relationship between music and emotions in children with hearing loss and cochlear implant users is considerably impaired, however the development of this skill could improve, to some extent, the identifying the emotional content of speech itself. In this sense, the present study aimed to teach the relations between the acoustic characteristics of musical excerpts and their respective emotional aspects to child cochlear implant users. To this purpose, eight children performed a four-step matching-to-sample (MTS) protocol in which this ability was tested and trained. In Step 1, participants learned to relate the dictated phrases "Point to Happy face" and "Point to Sad face" conditionally (in Portuguese) to happy and sad faces. In Step 2, it was tested whether participants could conditionally relate major and minor tonalities excerpts to happy or sad faces, respectively, even before exposure to the training protocols. In Step 3, children were taught to conditionally relate major and minor progressions to happy and sad faces, respectively. Finally, in Step 4, it was tested whether the proposed conditional training could provide any change in the discriminatory repertoire of child cochlear implant users with a new set of musical excerpts. In general, the results indicated that child cochlear implant users presented a statistically significant increase in their performances between Step 2 (pretest) and Step 4 (posttest). However, the performances were close to the level of chance for most participants in the Step 3 training blocks. In light of these results, evaluating other teaching procedures in future studies was suggested, such as fading and go/no-go procedures.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
Code Availability
Not applicable.
References
Abe, D., Arai, M., & Itokawa, M. (2017). Music-evoked emotions in schizophrenia. Schizophrenia Research, 185, 144–147.
Agustus, J. L., Mahoney, C. J., Downey, L. E., Omar, R., Cohen, M., White, M. J., Scott, S. K., Mancini, L., & Warren, J. D. (2015). Functional MRI of music emotion processing in frontotemporal dementia. Annals of the New York Academy of Sciences, 1337, 232–240. https://doi.org/10.1111/nyas.12620
Agrawal, D., Thorne, J. D., Viola, F. C., Timm, L., Debener, S., Büchner, A., Dengler, R., & Wittfoth, M. (2013). Electrophysiological responses to emotional prosody perception in cochlear implant users. NeuroImage: Clinical, 2, 229–238. https://doi.org/10.1016/j.nicl.2013.01.001
Almeida-Verdu, A. C. M., Bevilacqua, M. C., de Souza, D. D. G., & de Souza, F. C. (2009). Imitação vocal e nomeação de figuras em deficientes auditivos usuários de implante coclear: Estudo exploratório. Revista Brasileira de Análise do Comportamento, 5, 63–78. https://doi.org/10.18542/rebac.v5i1.722
Almeida-Verdu, A. C. M., & Golfeto, R. M. (2016). Stimulus control and verbal behavior: (in)Dependent relations in populations with minimal verbal repertoires. In J. C. Todorov (Ed.), Trends in Behavior Analysis (pp. 187–227). Technopolitik.
Almeida-Verdu, A. C. M., Huziwara, E. M., de Souza, D. G., de Rose, J. C., Bevilacqua, M. C., Lopes, J., Jr., Alves, C. O., & McIlvane, W. J. (2008). Relational learning in children with deafness and cochlear implants. Journal of the Experimental Analysis of Behavior, 89, 407–424. https://doi.org/10.1901/jeab.2008-89-407
Battaglini, M. P., Almeida-Verdu, A. C. M., & Bevilacqua, M. C. (2013). Aprendizagem via exclusão e formação de classes de equivalência em crianças com deficiência auditiva e implante coclear. Acta Comportamentalia, 21, 20–35.
Caldwell, M., Rankin, S. K., Jiradejvong, P., Carver, C., & Limb, C. J. (2015). Cochlear implant users rely on tempo rather than on pitch information during perception of musical emotion. Cochlear Implants International, 16, S114–S120. https://doi.org/10.1179/1467010015Z.000000000265
Carrigan, P. F., & Sidman, M. (1992). Conditional discrimination and equivalence relations: A theoretical analysis of control by negative stimuli. Journal of the Experimental Analysis of Behavior, 58, 183–204. https://doi.org/10.1901/jeab.1992.58-183
Cedro, A. M., & Huziwara, E. M. (2022). A sco** review of musical training with the methods of the experimental analysis of behavior. Trends in Psychology. https://doi.org/10.1007/s43076-022-00230-2
Chermak, G. D. (2010). Music and auditory training. The Hearing Journal, 63, 58. https://doi.org/10.1097/01.HJ.0000370863.67892.0d
Clark, G. (2004). Cochlear implants. In Speech processing in the auditory system (pp. 422–462). New York: Springer.
Corrigall, K. A., & Schellenberg, G. (2015). Predicting who takes music lessons: Parent and child characteristics. Frontiers in Psychology, 6, 282. https://doi.org/10.3389/fpsyg.2015.00282
Cunningham, J. G., & Sterling, R. S. (1988). Developmental change in the understanding of affective meaning in music. Motivation and Emotion, 12, 399–413. https://doi.org/10.1007/BF00992362
Dalla Bella, S., Peretz, I., Rousseau, L., & Gosselin, N. (2001). A developmental study of the affective value of tempo and mode in music. Cognition, 80, B1–B10. https://doi.org/10.1016/S0010-0277(00)00136-0
Debert, P., Huziwara, E. M., Faggiani, R. B., de Mathis, M. E. S., & McIlvane, W. J. (2009). Emergent conditional relations in a go/no-go procedure: Figure/ground and stimulus-position compound relations. Journal of the Experimental Analysis of Behavior, 92, 233–243. https://doi.org/10.1901/jeab.2009.92-233
Debert, P., Matos, M. A., & McIlvane, W. J. (2007). Conditional relations with compound abstract stimuli using a Go/No-Go procedure. Journal of the Experimental Analysis of Behavior, 87, 89–96. https://doi.org/10.1901/jeab.2007.46-05
Dolgin, K. G., & Adelson, E. H. (1990). Age changes in the ability to interpret affect in sung and instrumentally-presented melodies. Psychology of Music, 18, 87–98. https://doi.org/10.1177/0305735690181007
Fu, Q. J., Galvin, J. J., III, Wang, X., & Wu, J. L. (2015). Benefits of music training in Mandarin-speaking pediatric cochlear implant users. Journal of Speech, Language, & Hearing Research, 58, 163–169. https://doi.org/10.1044/2014_JSLHR-H-14-0127
Gangnon, L., & Peretz, I. (2003). Mode and tempo relative contributions too "happy-sad" judgments in equitone melodies. Cognition & Emotion, 17, 25–40. https://doi.org/10.1080/02699930302279
Gfeller, K. (2016). Music-based training for pediatric CI recipients: A systematic analysis of published studies. European Annals of Otorhinolaryngology, Head & Neck Diseases, 133, S50–S56. https://doi.org/10.1016/j.anorl.2016.01.010
Giannantonio, S., Polonenko, M. J., Papsin, B. C., Paludetti, G., & Gordon, K. A. (2015). Experience changes how emotion in music is judged: Evidence from children listening with bilateral cochlear implants, bimodal devices, and normal hearing. PloS One, 10, 1–29. https://doi.org/10.1371/journal.pone.0136685
Gregory, A. H., Worrall, L., & Sarge, A. (1996). The development of emotional responses to music in young children. Motivation & Emotion, 20, 341–348. https://doi.org/10.1007/BF02856522
Hopyan, T., Gordon, K. A., & Papsin, B. C. (2011). Identifying emotions in music through electrical hearing in deaf children using cochlear implants. Cochlear Implants International, 12, 21–26. https://doi.org/10.1179/146701010X12677899497399
Hopyan-Misakyan, T. M., Gordon, K. A., Dennis, M., & Papsin, B. (2009). Recognition of affective speech prosody and facial affect in deaf children with unilateral right cochlear implants. Child Neuropsychology, 15, 136–146. https://doi.org/10.1080/09297040802403682
Hunter, P. G., Schellenberg, E. G., & Schimmack, U. (2010). Feelings and perceptions of happiness and sadness induced by music: Similarities, differences, and mixed emotions. Psychology of Aesthetics, Creativity, & the Arts, 4, 47–56. https://doi.org/10.1037/a0016873
Jentschke, S., Koelsch, S., & Friederici, A. D. (2005). Investigating the relationship of music and language in children. Annals of the New York Academy of Sciences, 1060, 231–242. https://doi.org/10.1196/annals.1360.016
Juslin, P. N., & Laukka, P. (2003). Communication of emotions in vocal expression and music performance: Different channels, same code? Psychological Bulletin, 129, 770–814. https://doi.org/10.1037/0033-2909.129.5.770
Karam, M., Branje, C., Nespoli, G., Thompson, N., Russo, F. A., & Fels, D. I. (2010). The emoti-chair: An interactive tactile music exhibit. CHI'10 Extended Abstracts on Human Factors in Computing Systems, 3069–3074. https://doi.org/10.1145/1753846.1753919
Karam, M., Russo, F., Branje, C., Price, E., & Fels, D. I. (2008). Towards a model human cochlea: Sensory substitution for crossmodal audio-tactile displays. Proceedings of Graphics Interface, 267–274. https://doi.org/10.5555/1375714.1375759
Kastner, M. P., & Crowder, R. G. (1990). Perception of the major/minor distinction: IV. Emotional connotations in young children. Music Perception: An Interdisciplinary Journal, 8, 189–202. https://doi.org/10.2307/40285496
Kerer, M., Marksteiner, J., Hinterhuber, H., Kemmler, G., Bliem, H. R., & Weiss, E. M. (2014). Happy and sad judgements in dependence on mode and note density in patients with mild cognitive impairment and early-stage Alzheimer's disease. Gerontology, 60, 402–412.
Kim, S. J., Kim, E. Y., & Yoo, G. E. (2017). Music perception training for pediatric cochlear implant recipients ages 3 to 5 years: A pilot study. Music Therapy Perspectives, 35, 50–57. https://doi.org/10.1093/mtp/miw009
Koolagudi, S. G., & Rao, K. S. (2012). Emotion recognition from speech: A review. International Journal of Speech Technology, 15, 99–117. https://doi.org/10.1007/s10772-011-9125-1
Kratus, J. (1993). A developmental study of children's interpretation of emotion in music. Psychology of Music, 21, 3–19. https://doi.org/10.1177/030573569302100101
Kraus, N., & Skoe, E. (2009). New directions: Cochlear implants. Annals of the New York Academy of Sciences, 1169, 516–517. https://doi.org/10.1111/j.1749-6632.2009.04862.x
Laneau, J., Wouters, J., & Moonen, M. (2006). Improved music perception with explicit pitch coding in cochlear implants. Audiology & Neurotology, 11, 38–52. https://doi.org/10.1159/000088853
Lassaletta, L., Castro, A., Bastarrica, M., Pérez-Mora, R., Madero, R., Sarriá, J., & Gavilán, J. (2007). Does music perception have an impact on quality of life following cochlear implantation? Acta Oto-Laryngologica, 127, 682–686. https://doi.org/10.1080/00016480601002112
Lima, C. F., & Castro, S. L. (2011). Speaking to the trained ear: musical expertise enhances the recognition of emotions in speech prosody. Emotion, 11, 1021. https://doi.org/10.1037/a0024521
Lucchesi, F. D. M., Almeida-Verdu, A. C. M., Bolsoni-Silva, A. T., Buffa, M. J. M. B., & das Graças de Souza, D. (2022). Speech accuracy and reading in children with cochlear implants. The Psychological Record, 72(4), 697–711. https://doi.org/10.1007/s40732-022-00518-w
Lundqvist, D., Flykt, A., & Öhman, A. (1998). The Karolinska Directed Emotional Faces—KDEF. [Database record]. APA PsycTests. https://doi.org/10.1037/t27732-000
Luo, X., Fu, Q., & Galvin, J. J. (2007). Vocal emotion recognition by normal hearing listeners and cochlear implant users. Trends in Hearing, 11, 301–315. https://doi.org/10.1177/1084713807305301
Machado, A. R., & Borloti, E. B. (2009). Formation of functional classes of musical stimuli. Paidéia, 19, 47–58. https://doi.org/10.1590/S0103-863X2009000100007
McDermott, H. J. (2004). Music perception with cochlear implants: A review. Trends in Amplification, 8, 49–82. https://doi.org/10.1177/108471380400800203
Neves, A. J., Almeida-Verdu, A. C. M., Silva, L. T. N., Moret, A. L. M., & de Souza, D. G. (2023). Sentence comprehension and production in children with cochlear implants: Errorless procedures and equivalence-based instruction. Behavioral Interventions, 38(2), 376–400. https://doi.org/10.1002/bin.1922
Pallesen, K. J., Brattico, E., Bailey, C., Korvenoja, A., Koivisto, J., Gjedde, A., & Carlson, S. (2005). Emotion processing of major, minor, and dissonant chords: a functional magnetic resonance imaging study. Annals of the New York Academy of Sciences, 1060, 450–453.
Patel, A. D. (2011). Why would musical training benefit the neural encoding of speech? The OPERA hypothesis. Frontiers in Psychology, 2, 142. https://doi.org/10.3389/fpsyg.2011.00142
Peretz, I., Gagnon, L., & Bouchard, B. (1998). Music and emotion: Perceptual determinants, immediacy, and isolation after brain damage. Cognition, 68, 111–141. https://doi.org/10.1016/S0010-0277(98)00043-2
Petersen, B., Weed, E., Sandmann, P., Brattico, E., Hansen, M., Sørensen, S. D., & Vuust, P. (2015). Brain responses to musical feature changes in adolescent cochlear implant users. Frontiers in Human Neuroscience, 9, 7. https://doi.org/10.3389/fnhum.2015.00007
Picanço, C. R. F. (2017). Stimulus control (0.0.4.13). https://github.com/cpicanco/stimulus_control
Scherer, K. R. (1995). Expression of emotion in voice and music. Journal of Voice, 9, 235–248. https://doi.org/10.1016/S0892-1997(05)80231-0
Schmeling, P. (2011). Berklee music theory: Books 1 & 2. Berklee Press.
Shahin, A. J. (2011). Neurophysiological influence of musical training on speech perception. Frontiers in Psychology, 2, 126. https://doi.org/10.3389/fpsyg.2011.00126
Shirvani, S., Jafari, Z., Motasaddi Zarandi, M., Jalaie, S., Mohagheghi, H., & Tale, M. R. (2015). Emotional perception of music in children with bimodal fitting and unilateral cochlear implant. Annals of Otology, Rhinology & Laryngology, 125, 470–477. https://doi.org/10.1177/0003489415619943
Shirvani, S., Jafari, Z., Sheibanizadeh, A., Zarandy, M. M., & Jalaie, S. (2014). Emotional perception of music in children with unilateral cochlear implants. Iranian Journal of Otorhinolaryngology, 26, 225–233.
Sidman, M. (1987). Two choices are not enough. Behavior Analysis, 22, 11–18.
Sidman, M., & Stoddard, L. T. (1967). The effectiveness of fading in programming a simultaneous form discrimination for retarded children. Journal of the Experimental Analysis of Behavior, 10, 3–15. https://doi.org/10.1901/jeab.1967.10-3
Suzuki, M., Okamura, N., Kawachi, Y., Tashiro, M., Arao, H., Hoshishiba, T., & Yanai, K. (2008). Discrete cortical regions associated with the musical beauty of major and minor chords. Cognitive, Affective, & Behavioral Neuroscience, 8, 126–131. https://doi.org/10.3758/CABN.8.2.126
Terwogt, M. M., & Van Grinsven, F. (1991). Musical expression of mood states. Psychology of Music, 19, 99–109. https://doi.org/10.1177/0305735691192001
Tillmann, B., Gosselin, N., Bigand, E., & Peretz, I. (2012). Priming paradigm reveals harmonic structure processing in congenital amusia. Cortex, 48, 1073–1078. https://doi.org/10.1016/j.cortex.2012.01.001
van Zyl, M., & Hanekom, J. J. (2013). When “okay” is not okay: Acoustic characteristics of single-word prosody conveying reluctance. Journal of the Acoustical Society of America, 133, EL13–EL19. https://doi.org/10.1121/1.4769399
Weiss, M. W., Cirelli, L. K., McDermott, J. H., & Trehub, S. E. (2019). Development of consonance preferences in Western listeners. Journal of Experimental Psychology: General, 149, 634–649.
Yang, J., Liang, Q., Chen, H., Liu, Y., & Xu, L. (2019). Singing proficiency of members of a choir formed by prelingually deafened children with cochlear implants. Journal of Speech, Language, & Hearing Research, 62, 1561–1573. https://doi.org/10.1044/2019_JSLHR-H-18-0385
Zhou, L., Liu, F., Jiang, J., & Jiang, C. (2019). Impaired emotional processing of chords in congenital amusia: Electrophysiological and behavioral evidence. Brain & Cognition, 135, 103577.
Funding
RMR had a doctoral scholarship from the Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG, Grant #5.308/15; student ID #13514). This research was also supported by the National Institute of Science and Technology on Behavior, Cognition, and Teaching (INCT - ECCE), supported by Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP, Grant #2014/50909-8), Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES, Grant #88887.136407/2017-00), and CNPq (Grant #465686/2014-1). We thank Deisy de Souza, chairperson of INCT/ECCE, for her leadership and for encouraging the publication of this article.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics Approval
All procedures were approved by the Committee of Ethics of Universidade Federal de Minas Gerais (process number: CAAE 4508615.2.0000.5149).
Consent to Participate
Informed consent was obtained from all individual participants included in the study.
Consent for Publication
All the authors approve the manuscript and agree with its submission to the journal. The participants have consented to the submission of their individual data to the journal.
Conflicts of Interest/Competing Interests
There are no conflicts of interest to declare.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rodrigues, R.M., Almeida Verdu, A.C.M., Postalli, L.M.M. et al. Teaching Conditional Relations between Music Progressions and Faces for Child Cochlear Implant Users. Psychol Rec 73, 289–299 (2023). https://doi.org/10.1007/s40732-023-00544-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40732-023-00544-2