Abstract
This chapter reviews recent research from the author’s laboratory on habit and goal-direction in instrumental learning and then considers some of its implications for a general view of instrumental behavior and addiction. Results suggest that habit develops under conditions that allow the individual to pay less attention to its behavior, i.e., when the habit’s trigger cue reliably predicts the reward. Other results suggest that a behavior’s status as a habit is not necessarily fixed or permanent; several environmental manipulations can make a habitual behavior become goal-directed again. Habit is more context-specific than goal-direction. The perspective that emerges suggests that habit may have an important but perhaps more circumscribed role in instrumental behavior (and addiction) than might often be assumed. For example, drug seeking can appear adaptable and flexible because behaviors that are more distal to the goal (e.g., general search behaviors) may be goal-directed at the same time behaviors that are more proximal to the goal (e.g., actual drug-taking responses) are habitual. And individuals with substance use habits might not appear more habit-prone than controls when they are tested for habit in the context of the lab. These and other challenges that have been raised for the role of habit in addiction are discussed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adams, C. D. (1982). Variations in the sensitivity of instrumental responding to reinforcer devaluation. The Quarterly Journal of Experimental Psychology Section B, 34, 77–98.
Adams, C. D., & Dickinson, A. (1981). Instrumental responding following reinforcer devaluation. Quarterly Journal of Experimental Psychology, 33B, 109–122.
Balleine, B. W. (2019). The meaning of behavior: Discriminating reflex and volition in the brain. Neuron, 104, 47–62.
Balleine, B. W., & O’Doherty, J. P. (2010). Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology, 35, 48–69.
Balleine, B. W., Garner, C., Gonzalez, F., & Dickinson, A. (1995). Motivational control of heterogeneous instrumental chains. Journal of Experimental Psychology: Animal Behavior Processes, 21, 203–217.
Balleine, B. W., Liljeholm, M., & Ostlund, S. B. (2009). The integrative function of the basal ganglia in instrumental conditioning. Behavioural Brain Research, 199, 43–52.
Becchi, S., Hood, J., Kendig, M. D., Mohammadkhani, A., Shipman, M. L., Balleine, B. W., Borgland, S. L., & Corbit, L. H. (2022). Food for thought: Diet-induced impairments to decision-making and amelioration by N-acetylcysteine in male rates. Psychopharmacology, 239, 3495–3506.
Bouton, M. E. (2019). Extinction of instrumental (operant) learning: Interference, varieties of context, and mechanisms of contextual control. Psychopharmacology, 236, 7–19.
Bouton, M. E. (2021). Context, attention, and the switch between habit and goal-direction in behavior. Learning & Behavior, 49, 349–362.
Bouton, M. E., & Broomer, M. C. (2023). Learning to stop responding. Behavioural Processes, 206, 104830.
Bouton, M. E., & Schepers, S. T. (2015). Renewal after the punishment of free operant behavior. Journal of Experimental Psychology: Animal Learning and Cognition, 41, 81–90.
Bouton, M. E., Todd, T. P., Vurbic, D., & Winterbauer, N. E. (2011). Renewal after the extinction of free operant behavior. Learning & Behavior, 39, 57–67.
Bouton, M. E., Todd, T. P., & León, S. P. (2014). Contextual control of discriminated operant behavior. Journal of Experimental Psychology: Animal Learning and Cognition, 40, 92–105.
Bouton, M. E., Trask, S., & Carranza-Jasso, R. (2016). Learning to inhibit the response during instrumental (operant) extinction. Journal of Experimental Psychology: Animal Learning and Cognition, 42, 246–258.
Bouton, M. E., Broomer, M. C., Rey, C. N., & Thrailkill, E. A. (2020). Unexpected food outcomes can return a habit to goal-directed action. Neurobiology of Learning and Memory, 169, 107163.
Bouton, M. E., Allan, S. M., Tavakkoli, A., Steinfeld, M. R., & Thrailkill, E. A. (2021a). Effect of context on the instrumental reinforcer devaluation effect produced by taste aversion learning. Journal of Experimental Psychology: Animal Learning and Cognition, 47, 476–489.
Bouton, M. E., Maren, S., & McNally, G. P. (2021b). Behavioral and neurobiological mechanisms of Pavlovian and instrumental extinction learning. Physiological Reviews, 101, 611–681.
Broomer, M. C., & Bouton, M. E. (2023). A comparison of renewal, spontaneous recovery, and reacquisition after instrumental punishment and extinction. Learning & Behavior, 51, 262.
Camerer, C. L., & Li, X. (2022). Neural autopilot and context-sensitivity of habits. Current Opinion in Behavioral Sciences, 41, 185–190.
Collier, G. H. (1981). Determinants of choice. Nebraska Symposium on Motivation, 29, 69–127.
Colwill, R. M. (1991). Negative discriminative stimuli provide information about the identity of omitted response-contingent outcomes. Animal Learning & Behavior, 19, 326–336.
Colwill, R. M., & Rescorla, R. A. (1985a). Instrumental responding remains sensitive to reinforcer devaluation after extensive training. Journal of Experimental Psychology: Animal Behavior Processes, 11, 520–536.
Colwill, R. M., & Rescorla, R. A. (1985b). Postconditioning devaluation of a reinforcer affects instrumental responding. Journal of Experimental Psychology: Animal Behavior Processes, 11, 120–132.
Colwill, R. M., & Rescorla, R. A. (1988). The role of response-reinforcer associations increases throughout extended instrumental training. Animal Learning & Behavior, 16, 105–111.
Corbit, L. H., & Balleine, B. W. (2003). Instrumental and Pavlovian incentive processes have dissociable effects on components of a heterogeneous instrumental chain. Journal of Experimental Psychology: Animal Behavior Processes, 29, 99–106.
Corbit, L. H., Nie, H., & Janak, P. H. (2012). Habitual alcohol seeking: Time course and the contribution of subregions of the dorsal striatum. Biological Psychiatry, 72, 389–395.
Corbit, L. H., Chieng, B. C., & Balleine, B. W. (2014). Effects of repeated cocaine exposure on habit learning and reversal by N-acetylcysteine. Neuropsychopharmacology, 39, 1893–1901.
Coutureau, E., & Killcross, S. (2003). Inactivation of the infralimbic prefrontal cortex reinstates goal-directed responding in overtrained rats. Behavioural Brain Research, 146, 167–174.
Daw, N. D. (2015). Of goals and habits. National Academy of Sciences of the United States of America, 112, 13749.
Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8, 1704–1711.
de Wit, S., Kindt, M., Knot, S. L., Verhoeven, A. A. C., Robbins, T. W., Gasull-Camos, J., Evans, M., Mirza, H., & Gillan, C. M. (2018). Shifting the balance between goals and habits: Five failures in experimental habit induction. Journal of Experimental Psychology: General, 147, 1043–1065.
DeRusso, A., Fan, D., Gupta, J., Shelest, O., Costa, R. M., & Yin, H. H. (2010). Instrumental uncertainty as a determinant of behavior under interval schedules of reinforcement. Frontiers in Integrative Neuroscience, 4, 17.
Dezfouli, A., Lingawi, N. W., & Balleine, B. W. (2014). Habits as action sequences: hierarchical action control and changes in outcome value. Philosophical Transactions of the Royal Society B: Biological Sciences, 369, 20130482.
Dickinson, A. (1985). Actions and habits: The development of behavioural autonomy. Philosophical Transactions of the Royal Society of London. B, Biological Sciences, 308, 67–78.
Dickinson, A. (1989). Expectancy theory in animal conditioning. In S. B. Klein & R. R. Mowrer (Eds.), Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory (pp. 279–308). Erlbaum.
Dickinson, A., Nicholas, D. J., & Adams, C. D. (1983). The effect of the instrumental training contingency on susceptibility to reinforcer devaluation. The Quarterly Journal of Experimental Psychology, 35, 35–51.
Domjan, M. (1994). Formulation of a behavior system for sexual conditioning. Psychonomic Bulletin & Review, 1, 421–428.
Du, Y., Krakauer, J. W., & Haith, A. M. (2022). The relationship between habits and motor skills in humans. Trends in Cognitive Sciences, 26, 371–387.
Everitt, B. J., & Robbins, T. W. (2005). Neural systems of reinforcement for drug addiction: From actions to habits to compulsion. Nature Neuroscience, 8, 1481–1489.
Everitt, B. J., & Robbins, T. W. (2016). Drug addiction: Updating actions to habits to compulsions ten years on. Annual Review of Psychology, 67, 23–50.
Faure, A., Haberland, U., Condé, F., & El Massioui, N. (2005). Lesion to the nigrostriatal dopamine system disrupts stimulus-response habit formation. Journal of Neuroscience, 25, 2771–2780.
Faure, A., LeBlanc-Veyrac, P., & El Massoui, N. (2010). Dopamine agonists increase perseverative instrumental responses but do not restore habit formation in a rat model of Parkinsonism. Neuroscience, 168, 477–486.
Furlong, T. M., Jayaweera, H. K., Balleine, B. W., & Corbit, L. H. (2014). Binge-like consumption of a palatable food accelerates habitual control of behavior and is dependent on activation of the dorsolateral striatum. Journal of Neuroscience, 34, 5012–5022.
Furlong, T. M., Corbit, L. H., Brown, R. A., & Balleine, B. W. (2018). Methamphetamine promotes habitual action and alters the density of striatal glutamate receptor and vesicular proteins in dorsal striatum. Addiction Biology, 23, 857–867.
Garr, E., Bushra, B., Tu, N., & Delamater, A. R. (2020). Goal-directed control on interval schedules does not depend on the action–outcome correlation. Journal of Experimental Psychology: Animal Learning and Cognition, 46, 47–64.
Gremel, C. M., & Costa, R. M. (2013). Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nature Communications, 4, 2264–2276.
Gruber, A. J., & McDonald, R. J. (2012). Context, emotion, and the strategic pursuit of goals: Interactions among multiple brain systems controlling motivated behavior. Frontiers in Behavioral Neuroscience, 6, 60.
Hearst, E., & Peterson, G. B. (1973). Transfer of conditioned excitation and inhibition from one operant response to another. Journal of Experimental Psychology, 99, 360–368.
Hogarth, L. (2018). A critical review of habit theory of drug dependence. In B. Verplanken (Ed.), The psychology of habit: Theory, mechanisms, change, and contexts (pp. 325–342). Springer.
Hogarth, L. (2020). Addiction is driven by excessive goal-directed drug choice under negative affect: Translational critique of habit and compulsion theory. Neuropsychopharmacology, 45, 720–735.
Hogarth, L., & Chase, H. W. (2011). Parallel goal-directed and habitual control of human drug-seeking: Implications for dependence vulnerability. Journal of Experimental Psychology: Animal Behavior Processes, 37, 261–276.
Hogarth, L., Dickinson, A., Austin, A., Brown, C., & Duka, T. (2008). Attention and expectation in human predictive learning: The role of uncertainty. Quarterly Journal of Experimental Psychology, 61, 1658–1668.
Hogarth, L., Balleine, B. W., Corbit, L. H., & Killcross, S. (2013). Associative learning mechanisms underpinning the transition from recreational drug use to addiction. Annals of the New York Academy of Sciences, 1282, 12–24.
Hogarth, L., Lam-Cassettari, C., Pacitti, H., Currah, T., Mahlberg, J., Hartley, L., & Moustafa, A. (2019). Intact goal-directed control in treatment-seeking drug users indexed by outcome-devaluation and Pavlovian to instrumental transfer: Critique of habit theory. European Journal of Neuroscience, 50, 2513–2525.
Holland, P. C. (2004). Relations between Pavlovian-instrumental transfer and reinforcer devaluation. Journal of Experimental Psychology: Animal Behavior Processes, 30, 104–117.
Kaye, H., & Pearce, J. M. (1984). The strength of the orienting response during Pavlovian conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 10, 90–109.
Keramati, M., Dezfouli, A., & Piray, P. (2011). Speed/accuracy trade-off between the habitual and the goal-directed processes. PLoS Computational Biology, 7, e1002055.
Kosaki, Y., & Dickinson, A. (2010). Choice and contingency in the development of behavioral autonomy during instrumental conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 36, 334–342.
Le Pelley, M. E. (2004). The role of associative history in models of associative learning: A selective review and a hybrid model. Quarterly Journal of Experimental Psychology, 57B, 193–243.
Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014). Neural computations underlying arbitration between model-based and model-free learning. Neuron, 81, 687–699.
Mackintosh, N. J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276–298.
Nakajima, S., Tanaka, S., Urushihara, K., & Imada, H. (2000). Renewal of extinguished lever-press responses upon return to the training context. Learning and Motivation, 31, 416–431.
Nelson, A., & Killcross, S. (2006). Amphetamine exposure enhances habit formation. The Journal of Neuroscience, 26, 3805–3812.
Nelson, A. J. D., & Killcross, S. (2013). Accelerated habit formation following amphetamine exposure is reversed by D1, but enhanced by D2, receptor antagonists. Frontiers in Neuroscience, 7, 76.
Norman, D. A., & Shallice, T. (1986). Attention to action. In R. J. Davidson, G. E. Schwartz, & D. Shapiro (Eds.), Consciousness and self-regulation. Springer.
Olmstead, M. C., Lafond, M. V., Everitt, B. J., & Dickinson, A. (2001). Cocaine seeking by rats is a goal-directed action. Behavioral Neuroscience, 115, 394–402.
Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532–552.
Pearce, J. M., & Hall, G. (1982). Restoring the associability of a pre-exposed CS by a surprising event. Quarterly Journal of Experimental Psychology, 34B, 127–140.
Pearce, J. M., & Mackintosh, N. J. (2010). Two theories of attention: A review and a possible integration. In C. J. Mitchell & M. E. Le Pelley (Eds.), Attention and associative learning (pp. 11–39). Oxford University Press.
Perez, O. D., & Dickinson, A. (2020). A theory of actions and habits: The interaction of rate correlation and contiguity systems in free-operant behavior. Psychological Review, 127, 945–971.
Rescorla, R. A. (1990). Evidence for an association between the discriminative stimulus and the response-outcome association in instrumental learning. Journal of Experimental Psychology: Animal Behavior Processes, 16, 326–334.
Rescorla, R. A. (1993). Inhibitory associations between S and R in extinction. Animal Learning & Behavior, 21, 327–336.
Rescorla, R. A. (1997). Response inhibition in extinction. The Quarterly Journal of Experimental Psychology Section B, 50, 238–252.
Rosas, J. M., Todd, T. P., & Bouton, M. E. (2013). Context change and associative learning. Wiley Interdisciplinary Reviews: Cognitive Science, 4, 237–244.
Shipman, M. L., & Corbit, L. H. (2022). Diet-induced deficits in goal-directed control are rescued by agonism of group II metabotropic glutamate receptors in the dorsomedial striatum. Translational Psychiatry, 12, 42.
Shipman, M. L., Trask, S., Bouton, M. E., & Green, J. T. (2018). Inactivation of prelimbic and infralimbic cortex respectively affect expression of minimally-trained and extensively-trained goal-directed actions. Neurobiology of Learning and Memory, 155, 164–172.
Singer, B. F., Fadanelli, M., Kawa, A. B., & Robinson, T. E. (2018). Are cocaine-seeking “habits” necessary for the development of addiction-like behavior in rats? The Journal of Neuroscience, 38, 60–73.
Steinfeld, M. R., & Bouton, M. E. (2020). Context and renewal of habits and goal-directed actions after extinction. Journal of Experimental Psychology: Animal Learning and Cognition, 46, 408–421.
Steinfeld, M. R., & Bouton, M. E. (2021). Renewal of goal direction with a context change after habit learning. Behavioral Neuroscience, 135, 79–87.
Steinfeld, M. R., & Bouton, M. E. (2022). Inhibition in instrumental learning: Tests of response-specificity after feature-negative and extinction learning. Journal of Experimental Psychology: Animal Learning and Cognition, 48, 413–434.
Thorndike, E. L. (1911). Animal Intelligence. Macmillan.
Thrailkill, E. A., & Bouton, M. E. (2015). Contextual control of instrumental actions and habits. Journal of Experimental Psychology: Animal Learning and Cognition, 41, 69–80.
Thrailkill, E. A., & Bouton, M. E. (2016a). Extinction and the associative structure of heterogeneous instrumental chains. Neurobiology of Learning and Memory, 133, 61–68.
Thrailkill, E. A., & Bouton, M. E. (2016b). Extinction of chained instrumental behaviors: Effects of consumption extinction on procurement responding. Learning & Behavior, 44, 85–96.
Thrailkill, E. A., & Bouton, M. E. (2017). Effects of outcome devaluation on instrumental behaviors in a discriminated heterogeneous chain. Journal of Experimental Psychology: Animal Learning and Cognition, 43, 88–95.
Thrailkill, E. A., Trott, J. M., Zerr, C. L., & Bouton, M. E. (2016). Contextual control of chained instrumental behaviors. Journal of Experimental Psychology: Animal Learning and Cognition, 42, 401–414.
Thrailkill, E. A., Trask, S., Vidal, P., Alcalá, J. A., & Bouton, M. E. (2018). Stimulus control of actions and habits: A role for reinforcer predictability and attention in the development of habitual behavior. Journal of Experimental Psychology: Animal Learning and Cognition, 44, 370–384.
Thrailkill, E. A., Michaud, N., & Bouton, M. E. (2021). Reinforcer predictability and stimulus salience promote discriminated habit learning. Journal of Experimental Psychology: Animal Learning and Cognition, 47, 183–199.
Timberlake, W. (2001). Motivational modes in behavior systems. In R. R. Mowrer & S. B. Klein (Eds.), Handbook of contemporary learning theories (pp. 155–210). Erlbaum.
Todd, T. P. (2013). Mechanisms of renewal after the extinction of instrumental behavior. Journal of Experimental Psychology: Animal Behavior Processes, 39, 193–207.
Trask, S., Shipman, M. L., Green, J. T., & Bouton, M. E. (2020). Some factors that restore goal-direction to a habitual behavior. Neurobiology of Learning and Memory, 169, 107161.
Urcelay, G. P., & Jonkman, S. (2019). Delayed rewards facilitate habit formation. Journal of Experimental Psychology: Animal Learning and Cognition, 45, 413–421.
Vandaele, Y., & Ahmed, S. H. (2020). Habit, choice, and addiction. Neuropsychopharmacology, 46, 689–698.
Vandaele, Y., & Janak, P. H. (2023). Lack of action monitoring as a prerequisite for habitual and chunked behavior: Behavioral and neural correlates. iScience, 26, 105818.
Vandaele, Y., Pribut, H. J., & Janak, P. H. (2017). Lever insertion as a salient stimulus promoting insensitivity to outcome devaluation. Frontiers in Integrative Neuroscience, 11, 23.
Vandaele, Y., Vouillac-Mendoza, C., & Ahmed, S. H. (2019). Inflexible habitual decision-making during choice between cocaine and a nondrug alternative. Translational Psychiatry, 9, 109.
Vandaele, Y., Guillem, K., & Ahmed, S. H. (2020). Habitual preference for the nondrug reward in a drug choice setting. Frontiers in Behavioral Neuroscience, 14, 78.
Watson, P., O’Callaghan, C. O., Perkes, I., Bradfield, L., & Turner, K. (2022). Making habits measurable beyond what they are not: A focus on associative dual-process models. Neuroscience and Biobehavioral Reviews, 142, 104869.
Wilson, P. N., Boumphrey, P., & Pearce, J. M. (1992). Restoration of the orienting response to a light by a change in its predictive accuracy. The Quarterly Journal of Experimental Psychology, 44B, 17–36.
Wood, W., Quinn, J. M., & Kashy, D. A. (2002). Habits in everyday life: Thoughts, emotion, and action. Journal of Personality and Social Psychology, 83, 1281–1297.
Wood, W., Mazar, A., & Neal, D. T. (2022). Habits and goals in human behavior: Separate but interacting systems. Perspectives on Psychological Science, 17, 590–605.
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2006). Inactivation of dorsolateral striatum enhances sensitivity to changes in the action-outcome contingency in instrumental conditioning. Behavioural Brain Research, 166, 189–196.
Zapata, A., Minney, V. L., & Shippenberg, T. S. (2010). Shift from goal-directed to habitual cocaine seeking after prolonged experience in rats. Journal of Neuroscience, 30, 15457–15463.
Acknowledgements
Manuscript preparation and the research reported here were supported by Grant RO1 DA 033123 from the US National Institutes of Health. Thanks to Matt Broomer, Noelle Michaud, and Eric Thrailkill for their discussion.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Bouton, M.E. (2024). Situating Habit and Goal-Direction in a General View of Instrumental Behavior. In: Vandaele, Y. (eds) Habits. Springer, Cham. https://doi.org/10.1007/978-3-031-55889-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-55889-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-55888-7
Online ISBN: 978-3-031-55889-4
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)