SpringerLink
Log in

Involvement of Midbrain Dopamine Neuron Activity in Negative Reinforcement Learning in Mice

  • Original Article
  • Published:
Molecular Neurobiology Aims and scope Submit manuscript

A Correction to this article was published on 21 October 2021

This article has been updated

Abstract

The activity of the midbrain dopamine system reflects the valence of environmental events and modulates various brain structures to modify an organism’s behavior. A series of recent studies reported that the direct and indirect pathways in the striatum are critical for instrumental learning, but the dynamic changes in dopamine neuron activity that occur during negative reinforcement learning are still largely unclear. In the present study, by using a negative reinforcement learning paradigm employing foot shocks as aversive stimuli, bidirectional changes in substantia nigra pars compacta (SNc) dopamine neuron activity in the learning and habituation phases were observed. The results showed that in the learning phase, before mice had mastered the skill of esca** foot shocks, the presence of foot shocks induced a transient reduction in the activity of SNc dopamine neurons; however, in the habituation phase, in which the learned skill was automated, it induced a transient increase. Microinjection of a dopamine D1 receptor (D1R) or D2 receptor (D2R) antagonist into the dorsomedial striatum (DMS) significantly impaired learning behavior, suggesting that the modulatory effects of dopamine on both the direct and indirect pathways are required. Moreover, during the learning phase, excitatory synaptic transmission to DMS D2R-expressing medium spiny neurons (D2-MSNs) was potentiated. However, upon completion of the learning and habituation phases, the synapses onto D1R-expressing medium spiny neurons (D1-MSNs) were potentiated, and those onto D2-MSNs were restored to normal levels. The bidirectional changes in both SNc dopamine neuron activity and DMS synaptic plasticity might be the critical neural correlates for negative reinforcement learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

All data generated during this study are available from the corresponding author on request.

Code Availability

Not applicable.

Change history

Abbreviations

SNc:

Substantia nigra pars compacta

DA:

Dopamine

DMS:

Dorsomedial striatum

MSNs:

Medium spiny neurons

D1R:

Dopamine D1 receptor

D2R:

Dopamine D2 receptor

ERF:

The event-related fluorescence

sEPSCs:

Spontaneous excitatory postsynaptic currents

References

  1. Nakanishi S, Hikida T, Yawata S (2014) Distinct dopaminergic control of the direct and indirect pathways in reward-based and avoidance learning behaviors. Neurosci 282:49–59. https://doi.org/10.1016/j.neuroscience.2014.04.026

    Article  CAS  Google Scholar 

  2. Graybiel AM (2008) Habits, rituals, and the evaluative brain. Ann Rev Neurosci 31:359–387. https://doi.org/10.1146/annurev.neuro.29.051605.112851

    Article  CAS  PubMed  Google Scholar 

  3. Gerfen CR, Surmeier DJ (2011) Modulation of striatal projection systems by dopamine. Ann Rev Neurosci 34:441–466. https://doi.org/10.1146/annurev-neuro-061010-113641

    Article  CAS  PubMed  Google Scholar 

  4. Bromberg-Martin ES, Matsumoto M, Hikosaka O (2010) Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68:815–834. https://doi.org/10.1016/j.neuron.2010.11.022

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. **ao X, Deng H, Furlan A, Yang T, Zhang X, Hwang GR, Tucciarone J, Wu P et al (2020) A genetically defined compartmentalized striatal direct pathway for negative reinforcement. Cell 183:211–227. https://doi.org/10.1016/j.cell.2020.08.032

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Costa RM (2007) Plastic corticostriatal circuits for action learning: what’s dopamine got to do with it? Ann N Y Acad Sci 1104:172–191. https://doi.org/10.1196/annals.1390.015

    Article  CAS  PubMed  Google Scholar 

  7. Yin HH, Mulcare SP, Hilário MRF, Clouse E, Holloway T, Davis MI, Hansson AC, Lovinger DM et al (2009) Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat Neurosci 12:333–341. https://doi.org/10.1038/nn.2261

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Kravitz AV, Kreitzer AC (2012) Striatal mechanisms underlying movement, reinforcement, and punishment. Physiol 27(3):1–20. https://doi.org/10.1152/physiol.00004.2012

    Article  Google Scholar 

  9. Cui G, Jun SB, ** X, Pham MD, Vogel SS, Lovinger DM, Costa RM (2013) Concurrent activation of striatal direct and indirect pathways during action initiation. Nature 494(7436):238–242. https://doi.org/10.1038/nature11846

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Tecuapetla F, ** X, Lima SQ, Costa RM (2016) Complementary contributions of striatal projection pathways to action initiation and execution. Cell 166:703–715. https://doi.org/10.1016/j.cell.2016.06.032

    Article  CAS  PubMed  Google Scholar 

  11. Shan Q, Ge M, Christie MJ, Balleine BW (2014) The acquisition of goal-directed actions generates opposing plasticity in direct and indirect pathways in dorsomedial striatum. J Neurosci 34:9196–9201. https://doi.org/10.1523/JNEUROSCI.0313-14.2014

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Fiorillo CD, Yun SR, Song MR (2013) Diversity and homogeneity in responses of midbrain dopamine neurons. J Neurosci 33(11):4693–4709. https://doi.org/10.1523/JNEUROSCI.3886-12.2013

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Suri RE, Schultz W (1998) Learning of sequential movements by neural network model with dopamine-like reinforcement signal. Exp Brain Res 121:350–354. https://doi.org/10.1007/s002210050467

    Article  CAS  PubMed  Google Scholar 

  14. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Sci 275:1593–1599. https://doi.org/10.1126/science.275.5306.1593

    Article  CAS  Google Scholar 

  15. Zhang X, Kim J, Tonegawa S (2020) Amygdala reward neurons form and store fear extinction memory. Neuron 105:1077–1093. https://doi.org/10.1016/j.neuron.2019.12.025

    Article  CAS  PubMed  Google Scholar 

  16. Yu T, Li Y, Hu Q, Wang F, Yuan S, Li C, Li JP, Cui JL et al (2020) Ketamine contributes to the alteration of Ca2+ transient evoked by behavioral tests in the prelimbic area of mPFC: a study on chronic CORT-induced depressive mice. Neurosci Lett 735:135220. https://doi.org/10.1016/j.neulet.2020.135220

    Article  CAS  PubMed  Google Scholar 

  17. Nasehi M, Hasanvand S, Khakpai F, Zarrindast MR (2019) The effect of CA1 dopaminergic system on amnesia induced by harmane in mice. Acta Neurol Belg 119:369–377. https://doi.org/10.1007/s13760-018-0926-8

    Article  PubMed  Google Scholar 

  18. Piri M, Ayazi E, Zarrindast MR (2013) Involvement of the dorsal hippocampal dopamine D2 receptors in histamine-induced anxiogenic-like effects in mice. Neurosci Lett 550:139–144. https://doi.org/10.1016/j.neulet.2013.07.009

    Article  CAS  PubMed  Google Scholar 

  19. Yao L, Li Y, Qian Z, Wu M, Yang H, Chen N (2019) Loss of control over mild aversive events produces significant helplessness in mice. Behav Brain Res 376:112173. https://doi.org/10.1016/j.bbr.2019.112173

    Article  PubMed  Google Scholar 

  20. Wang XY, Qiao YH, Dai ZH, Sui N, Shen F, Zhang JJ, Liang J (2019) Medium spiny neurons of the anterior dorsomedial striatum mediate reversal learning in a cell-type-dependent manner. Brain Struct Funct 224:419–434. https://doi.org/10.1007/s00429-018-1780-4

    Article  CAS  PubMed  Google Scholar 

  21. Huang S, Uusisaari MY (2013) Physiologycal temperature during brain slicing enhances the quality of acute slice preparations. Front Cellular Neurosci 7:48. https://doi.org/10.3389/fncel.2013.00048

    Article  Google Scholar 

  22. Cadwell CR, Palasantza A, Jiang X, Berens P, Deng Q, Yilmaz M, Reimer J, Shenet S et al (2016) Electrophysiological, transcriptomic and morphologic profiling of single neurons using Patch-seq. Nat Biotechnol 34:199–203. https://doi.org/10.1038/nbt.3445

    Article  CAS  PubMed  Google Scholar 

  23. Zhong WX, Li Y, Feng QR, Luo MM (2017) Learning and stress shape the reward response patterns of serotonin neurons. J Neurosci 37:8863–8875. https://doi.org/10.1523/JNEUROSCI.1181-17.2017

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Wang DQ, Li Y, Feng QR, Guo QC, Zhou JF, Luo MM (2017) Learning shapes the aversion and reward responses of lateral habenula neurons. Elife 6:1–20. https://doi.org/10.7554/elife.23045

    Article  CAS  Google Scholar 

  25. Daubner SC, Le T, Wang S (2011) Tyrosine hydroxylase and regulation of dopamine synthesis. Arch Biochem Biophys 508:1–12. https://doi.org/10.1016/j.abb.2010.12.017

    Article  CAS  PubMed  Google Scholar 

  26. Gunaydin LA, Grosenick L, Finkelstein JC, Kauvar IV, Fenno LE, Adhikari A, Lammel S, Mirzabekov JJ et al (2014) Natural neural projection dynamics underlying social behavior. Cell 157:1535–1551. https://doi.org/10.1016/j.cell.2014.05.017

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Gallistel CR, Fairhurst SF, Balsam P (2004) The learning curve: implications of a quantitative analysis. PNAS 101:13124–13131. https://doi.org/10.1073/pnas.0404965101

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Choi WY, Balsam PD, Horvitz JC (2005) Extended habit training reduces dopamine mediation of appetitive response expression. J Neurosci 25:6729–6733. https://doi.org/10.1523/JNEUROSCI.1498-05.2005

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Costa RM, Cohen D, Nicolelis MAL (2004) Differential corticostriatal plasticity during fast and slow motor skill learning in mice. Curr Biol 14:1124–1134. https://doi.org/10.1016/j.cub.2004.06.053

    Article  CAS  PubMed  Google Scholar 

  30. André VM, Cepeda C, Cummings DM, Jocoy EL, Fisher YE, Yang XW, Levine MS (2010) Dopamine modulation of excitatory currents in the striatum is dictated by the expression of D1 or D2 receptor and modified by endocannabinoids. Eur J Neurosci 31:14–28. https://doi.org/10.1111/j.1460-9568.2009.07047.x

    Article  PubMed  Google Scholar 

  31. Shen W, Flajolet M, Greengard P, Surmeier DJ (2008) Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321:848–851. https://doi.org/10.1126/science.1160575

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Ade KK, Wan Y, Chen M, Gloss B, Calakos N (2011) An improved BAC transgenic fluorescent reporter line for sensitive and specific identification of striatonigral medium spiny neurons. Front Syst Neurosci 5:1–9. https://doi.org/10.3389/fnsys.2011.00032

    Article  CAS  Google Scholar 

  33. Heintz N (2002) BAC to the future: the use of BAC transgenic mice for neuroscience research. Nat Rev Neurosci 2:861–870. https://doi.org/10.1038/35104049

    Article  CAS  Google Scholar 

  34. Cepeda C, AndréVM YI, Wu N, Kleiman-Weiner M, Levine MS (2008) Differential electrophysiological properties of dopamine D1 and D2 receptor-containing striatal medium-sized spiny neurons. Eur J Neurosci 27:671–682. https://doi.org/10.1111/j.1460-9568.2008.06038.x

    Article  PubMed  Google Scholar 

  35. Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1–27. https://doi.org/10.1152/jn.1998.80.1.1

    Article  CAS  PubMed  Google Scholar 

  36. Higley MJ, Sabatini BL (2010) Competitive regulation of synaptic Ca2+ influx by D2 dopamine and A2A adenosine receptors. Nat Neurosci 13:958–966. https://doi.org/10.1038/nn.2592

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Tritsch NX, Sabatini BL (2012) Dopaminergic modulation of synaptic transmission in cortex and striatum. Neuron 76:33–50. https://doi.org/10.1016/j.neuron.2012.09.023

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Hikosaka O, Takikawa Y, Kawagoe R (2000) Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol Rev 80:953–978. https://doi.org/10.1152/physrev.2000.80.3.953

    Article  CAS  PubMed  Google Scholar 

  39. Mink JW (2003) The Basal Ganglia and involuntary movements: impaired inhibition of competing motor patterns. Arch Neurol 60:1365–1368. https://doi.org/10.1001/archneur.60.10.1365

    Article  PubMed  Google Scholar 

  40. Tepper JM, Wilson CJ, Koós T (2008) Feedforward and feedback inhibition in neostriatal GABAergic spiny neurons. Brain Res Rev 58:272–281. https://doi.org/10.1016/j.brainresrev.2007.10.008

    Article  CAS  PubMed  Google Scholar 

  41. Tecuapetla F, Koo´s T, Tepper JM, Kabbani N, Yeckel MF (2009) Differential dopaminergic modulation of neostriatal synaptic connections of striatopallidal axon collaterals. J Neurosci 29:8977–8990. https://doi.org/10.1523/JNEUROSCI.6145-08.2009

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. André VM, Cepeda C, Cummings DM, Jocoy EL, Fisher YE, Yang XW, Levine MS (2010) Dopamine modulation of excitatory currents in the striatum is dictated by the expression of D1 or D2 receptors and modified by endocannabinoids. Eur J Neurosci 31:14–28. https://doi.org/10.1111/j.1460-9568.2009.07047.x

    Article  PubMed  Google Scholar 

  43. ** X, Costa RM (2015) Sha** action sequences in basal ganglia circuits. Curr Opin Neurobiol 33:188–196. https://doi.org/10.1016/j.conb.2015.06.011

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Geddes CE, Li H, ** X (2018) Optogenetic Editing Reveals the Hierarchical Organization of Learned Action Sequences. Cell 174:32–43. https://doi.org/10.1016/j.cell.2018.06.012

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank Dr. Zhiqiang Liu of Shaanxi Normal University for expertise with electrophysiological recording.

Funding

This research was supported by the National Natural Science Foundation of China (Project No. 81971285; 11727813; 82071516), and by the Fundamental Research Funds for the Central Universities (Project No. GK202005001, GK202105001), Shaanxi Normal University.

Author information

Author notes

  1. The authors, Zhijun Diao and Li Yao, contribute equally to this paper.

Authors and Affiliations

  1. Key Laboratory of Modern Teaching Technology, Ministry of Education, Shaanxi Normal University, **’an, 710062, Shaanxi, China

    Zhijun Diao, Li Yao, Meilin Wu, Yuanyuan Di, Zhaoqiang Qian, Chunling Wei, Yingxun Liu & Wei Ren

  2. College of Life Sciences, Shaanxi Normal University, **’an, 710119, Shaanxi, China

    Qiangqiang Cheng & Yingfang Tian

  3. College of Psychology, Shaanxi Normal University, **’an, 710062, Shaanxi, China

    Wei Ren

  4. Faculty of Table Tennis, Badminton and Tennis, Chengdu Sport University, 610041, Chengdu, China

    Li Yao

Authors
  1. Zhijun Diao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiangqiang Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Meilin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuanyuan Di
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhaoqiang Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chunling Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yingxun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yingfang Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Wei Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: WR. Methodology: ZJD, LY, MLW, and YYD. Validation: WR, YFT, ZJD, LY, and YX-L. Formal analysis: WR, YFT, and ZJD. Investigation: ZJD, ZQQ, QQC, and CLW. Resources: YX-L, WR, and YFT. Writing, original draft: WR and ZJD. Writing, review and editing: WR, YFT, ZJD, LY, MLW, YYD, and QQC. Supervision: WR and YFT. Funding acquisition: WR and YFT. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Wei Ren.

Ethics declarations

Ethics Approval

All the experimental procedures were approved by the Medicine Animal Care and Use Committee of Shaanxi Normal University conformed to the Guide for the National Institutional Animal Care and the guidelines published in the NIH Guide for the Care and Use of Laboratory Animals. All efforts were made to minimize the number of animals and their suffering.

Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Diao, Z., Yao, L., Cheng, Q. et al. Involvement of Midbrain Dopamine Neuron Activity in Negative Reinforcement Learning in Mice. Mol Neurobiol 58, 5667–5681 (2021). https://doi.org/10.1007/s12035-021-02515-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12035-021-02515-6

Keywords

Search

Navigation