1. Lecture 17: Instrumental Conditioning (Associative Structures) Learning, Psychology 5310 Spring, 2015 Professor Delamater

2. Instrumental Learning: Associative Structures 1.S – R association 2.R – O association 3.S – O association 4.S – [R–O] association Lots of different types of learning could occur and account for why the animal increases its responding with training. S – R – O (3-term Contingency): What associations do they learn?

3. Thorndike’s Law of Effect (revisited) Cats learned to escape puzzle box to obtain food reward. Thorndike assumed this was caused by the development of an S-R association. Reinforcement “stamped in” this association, without itself being learned about.

4. Evidence for R – O Associations in Instrumental Learning Reward Devaluation Experiment During the test phase, subjects chose to work for the outcome that had not been devalued. This implies that they “knew” what they were working for, i.e., they acquired an R – O association. R1-O1 Devalue O1R1 vs R2(R1 < R2) R2-O2 No Deval of O2 R1 – TobaccoDevalue TobaccoR1 vs R2 R2 – ChocolatNo Deval of Choc Instr Train Outcome DevalTest Hogarth & Chase, 2011 Study

5. Evidence for R – O Associations in Instrumental Learning Reward Devaluation Experiment During the test phase, subjects chose to work for the outcome that had not been devalued. This implies that they “knew” what they were working for, i.e., they acquired an R – O association. R1-O1 Devalue O1R1 vs R2(R1 < R2) R2-O2 No Deval of O2 R1 – TobaccoDevalue TobaccoR1 vs R2 R2 – ChocolatNo Deval of Choc Instr Train Outcome DevalTest Hogarth & Chase, 2011 Study

6. Evidence for R – O & S – R Associations in Instrumental Learning Adams (1982) Experiment: Extensive vs Limited training One set of groups given 100 Lever-Pellet rewards, another given 500. Lever Press – Pellets Instr Train (100 or 500) Outcome DevalTest

7. Evidence for R – O & S – R Associations in Instrumental Learning Adams (1982) Experiment: Extensive vs Limited training One set of groups given 100 Lever-Pellet rewards, another given 500. One of each set then was given pellet devaluation, and the other no pellet devaluation. Lever Press – Pellets Pellets – IllnessLever Press ? Lever Press – Pellets Pellets | IllnessLever Press ? Instr Train (100 or 500) Outcome DevalTest

8. Evidence for R – O & S – R Associations in Instrumental Learning Adams (1982) Experiment: Extensive vs Limited training One set of groups given 100 Lever-Pellet rewards, another given 500. One of each set then was given pellet devaluation, and the other no pellet devaluation. Interestingly, the devaluation effect goes away with extensive instrumental training. This suggests that as the instrumental response becomes “habitual” the S-R association more strongly controls the response. However, with more limited instrumental training, the response is more strongly controlled by the R-O association. Lever Press – PelletsPellets – IllnessLever Press ? Lever Press – PelletsPellets | IllnessLever Press ? Instr Train (100 or 500) Outcome DevalTest

9. Evidence for S – O Associations in Instrumental Learning 1.Rescorla & Solomon (1967) Two Process Theory: S – O & S – R associations are learned. The stimulus (S) comes to “motivate” responding. It does this because S associates with the emotional aspects of the reinforcing outcome.

10. Evidence for S – O Associations in Instrumental Learning 1.Rescorla & Solomon (1967) Two Process Theory: S – O & S – R associations are learned. The stimulus (S) comes to “motivate” responding. It does this because S associates with the emotional aspects of the reinforcing outcome. Pavlovian – instrumental transfer test The Pavlovian CS (Tone) increases instrumental lever pressing when it is paired with food. But it decreases instrumental lever pressing when it is paired with foot shock (CER).

11. Evidence for S – O Associations in Instrumental Learning 1.Rescorla & Solomon (1967) Two Process Theory: S – O & S – R associations are learned. 2.Reward-specific Outcome expectancies: S – O associations can motivate instrumental responding because the stimulus associates with the specific features of the reinforcing outcome. The Pavlovian CSs exert a highly specific effect on instrumental responding. This implies that each CS associated with the sensory-specific features of the Outcome. R1-O1 CS1-O1 CS1: R1 vs R2(R1 > R2) R2-O2 CS2-O2 CS2: R1 vs R2(R1 < R2) R1 vs R2(R1 = R2) Instr Train Pav Train Test Kruse et al, 1983 Study

12. Evidence for S – [R – O] Associations in Instrumental Learning S1: R1-O1, R2-O2 Tone: R1-Pel, R2-Sucr R1 = Left lever press R2 = Right lever press Instr Train Outcome DevalTest Colwill & Rescorla, 1990 Study

13. Evidence for S – [R – O] Associations in Instrumental Learning S1: R1-O1, R2-O2 S2: R1-O2, R2-O1 Tone: R1-Pel, R2-Sucr Light: R1-Sucr, R2-Pel Instr Train Outcome DevalTest Colwill & Rescorla, 1990 Study Now, each of two Rs are reinforced with each of the two Os, but different R-O relations are signaled by different Ss.

14. Evidence for S – [R – O] Associations in Instrumental Learning S1: R1-O1, R2-O2Devalue O1 S2: R1-O2, R2-O1No Deval of O2 Tone: R1-Pel, R2-SucrPel – Illness Light: R1-Sucr, R2-PelSucr – No Illness Instr Train Outcome DevalTest Colwill & Rescorla, 1990 Study What would be the effect of devaluing one of the reinforcing outcomes?

15. Evidence for S – [R – O] Associations in Instrumental Learning S1: R1-O1, R2-O2Devalue O1S1: R1 vs R2 S2: R1-O2, R2-O1No Deval of O2S2: R1 vs R2 Tone: R1-Pel, R2-SucrPel – IllnessTone: R1 vs R2 Light: R1-Sucr, R2-PelSucr – No IllnessLight: R1 vs R2 Instr Train Outcome DevalTest Colwill & Rescorla, 1990 Study This was assessed in a test phase conducted under extinction conditions….

16. Evidence for S – [R – O] Associations in Instrumental Learning The rats selectively avoided the R when it signaled the devalued O in the presence of a specific S. The behavior must have been controlled by hierarchical S-[R-O] associations. S1: R1-O1, R2-O2Devalue O1S1: R1 vs R2(S1: R1 < R2) S2: R1-O2, R2-O1No Deval of O2S2: R1 vs R2(S2: R1 > R2) Tone: R1-Pel, R2-SucrPel – IllnessTone: R1 vs R2 Light: R1-Sucr, R2-PelSucr – No IllnessLight: R1 vs R2 Instr Train Outcome DevalTest Colwill & Rescorla, 1990 Study