1. 18 Actions, Habits, and the Cortico-Striatal System

2. To appreciate the basic problems addressed in this chapter, you might think back to when you initially learned to drive a car. In order to competently drive you had to learn and coordinate many complex behaviors, such as: Insert the ignition key Turn on the ignition key Release the handbrake Put the car in gear Put your foot on the accelerator Generate just the right amount of gas Adjust the steering wheel to maintain or change the trajectory Make proper fine-grain adjustments to oncoming traffic Apply just the right amount of pressure on the brake to stop Some Behaviors Are Modified by the Outcomes They Produce

3. Most psychologists use the term instrumental learning or instrumental behavior when referring to the study of how behavior is modified by the outcome it produces. This term recognizes that our behaviors can be viewed as instruments that can change or modify our environments. For example, when you turn the ignition key, the engine starts. The Concept of Instrumental Behavior

4. The Scientific Study of Instrumental Behavior Began with E.L. Thorndike

5. The essence of Thorndike’s theory is that outcomes produced by behavior ultimately adapt the animal to the situation by strengthening and weakening existing stimulus–response (S–R) connections. Outcomes that reward behavior strengthen S–R connections; nonrewarding outcomes weaken connections. Thorndike’s Law of Effect

6. According to E.C. Tolman: Instrumental behaviors are organized around goals and mediated by expectancies. An expectancy is a three-term association (S1–R–S2) that includes a representation of the stimulus situation (S1) that preceded the response, a representation of the response (R), and a representation of the outcome (S2) produced by the response. Tolman’s Cognitive Expectancy Theory

7. Comparison of the S–R Habit versus Expectancy Representation of the Cat’s Solution to the Puzzle Box

8. Instrumental Behaviors Can Be Classified as Either Actions or Habits Actions and habits differ on four dimensions.

9. The reward devaluation strategy centers on changing the value of the outcome after the animal has learned to solve the problem. How Can You Tell If an Instrumental Behavior Is an Action or a Habit? The Reward Devaluation Strategy

10. The Rat’s Lever-Pressing Response Becomes a Habit with Extensive Training Instrumental behaviors are initially controlled by the action system. This conclusion is based on the fact that after limited training the animals behavior is reduced following reward devaluation. However, with extensive practice the behaviors become habits. This conclusion is based on the fact that after extensive training, reward devaluation has no effect compared to the control condition.

11. Conceptual Model for Actions and Habits

12. Actions versus Habits Action and habit systems compete for control. Once an instrumental response is well learned, the two systems can work synergistically to support the same behavior, with the habit system dominating. However, when the contingencies are reversed, the action system attempts to rapidly adjust to the system but the output from the habit system can interfere with adaptations. Example: An American crossing the street in England has to overcome the habit of looking to the left before crossing.

13. The Acquisition of Instrumental Behaviors Depends on the Striatum Rats with damage to the striatum could not learn that immediately turning left or right to enter the adjacent arm would be rewarded.

14. The Striatum Integrates Information from Many Brain Regions to Create Instrumental Behaviors

15. Neural Support for the Action System: Dorsomedial Striatum Supports Actions Rats with damage to the dorsomedial striatum are not sensitive to reward devaluation.

16. Neural Support for the Action System: The Basolateral Amygdala Attaches Value Outcomes The amygdala is critical for attaching value to a particular outcome. Primates and rodents with damage to this region learn simple discriminations but are insensitive to reward devaluation. Sham Amygdala Con Deval Number of Response/min

17. The prelimbic region is critical in the acquisition of the associations that support an action. We know this because when this region is damaged prior to training, reward devaluation has no effect, meaning the behavior is a habit. The prelimbic region is not the site in the brain where these associations are stored. We know this because if this region is damaged after training, rats are sensitive to reward devaluation. Neural Support for Actions: Prelimbic Prefrontal Cortex Supports Action Learning but Is Not a Storage Site Prelimbic Prior After Con Deval Number of Response/min

18. Damage to the dorsolateral striatum prevents the development of habits. Following extensive lever press training, normal rats are insensitive to reward devaluation. However, rats with damage to the dorsolateral striatum are still sensitive to devaluation, indicating that the behavior never became a habit. Neural Support for Habits: Dorsolateral Striatum DLS damage Control DLS lesion Con Deval Number of Response/min

19. Damage to this region prior to training prevents instrumental behaviors from becoming habits. Rats with such damage remain sensitive to reward devaluation even after extensive training. Moreover, if this region is damaged after rats have be extensively trained and the instrumental response has become a habit, the rats again become sensitive to reward devaluation. All this suggests that with extensive training the infralimbic region comes to suppress the output of the action system. Infralimbic Medial Prefrontal Cortex Suppresses the Action System Infralimbic Prior After Con Deval Number of Response/min

20. The Contributions of the Prelimbic and Infralimbic Regions of the Prefrontal Cortex to Instrumental Behavior

21. The Contributions of the Prelimbic and Infralimbic Regions of the Medial Prefrontal Cortex to Instrumental Behavior Summary Prelimbic prefrontal cortex contributes to creating action based behavior. However, its not a storage site for the action pattern because damage to it after the action has been learned does not affect animals sensitive to reward devaluation. Infralimbic prefrontal cortex suppresses the action system. If this region is damaged before training the animal never becomes insensitive to reward devaluation even after extensive training. Moreover after extensive training, when the behavior is normally a habit, if the infralimbic region is damaged it again becomes action—sensitive to reward devaluation.

22. The Mesolimbic Dopamine System and the Dopamine Theory of Reinforcement (A) The mesolimbic dopamine system. The dopamine neurons are located in the ventral tegmental area (VTA) and their fibers project into the nucleus accumbens. (B) The dopamine theory of reinforcement. An outcome–reward has two functions: (1) It generates a representation (O), and (2) it activates dopamine neurons in the VTA that release dopamine. This strengthens synaptic connections between the representations of the stimulus (S) complex and the response (R) and perhaps between the representations of the response and the outcome.

23. The Incentive Salience Hypothesis of How Rewards Influence Instrumental Behavior The incentive salience hypothesis assumes that the reward turns on dopamine neurons in the VTA. In the normal sequence of events that establish instrumental behavior, the stimulus not only can associate with the response, it also can become associated with the incentive properties of the outcome. The stimulus itself can Then elicit strong urges or wants that lead the individual to seek out the reward.

24. Neural Support for Action and Habit Systems: Summary Instrumental Behavior Action System Dorsomedial striatum Basolateral amygdala Prelimbic prefrontal cortex Habit System Dorsolateral striatum Infralimbic prefrontal cortex Mesolimbic dopamine system