1. More Instrumental (Operant) Conditioning

2. B.F. Skinner Coined the term ‘Operant conditioning’ Coined the term ‘Operant conditioning’ The animal operates on the environment The animal operates on the environment Pioneered the use of the free operants Pioneered the use of the free operants Theory-Free Theory-Free

3. The Skinner Box Automatic Automatic Easy measurements that can be compared across species Easy measurements that can be compared across species

5. Operant Strengthened Bite Groom Lick Rear Push Lever Reinforcer

6. Techniques Shaping: Successive approximations Shaping: Successive approximations Require closer and closer appoximations to the target behaviour Require closer and closer appoximations to the target behaviour Secondary Reinforcers: Secondary Reinforcers: Stimuli accompanying reinforcer delivery Stimuli accompanying reinforcer delivery Marking: Marking: Feedback that a response had occurred Feedback that a response had occurred

7. Key concepts and terms Three term contingency Three term contingency Discriminative stimulus Discriminative stimulus Operant Operant Consequence Consequence Acquisition Acquisition Extinction Extinction Spontaneous recovery Spontaneous recovery Generalization Generalization Conditioned reinforcement Conditioned reinforcement Response chains Response chains

8. Other Similarities = bar press= food Perfect contingency Strong Responding Degraded contingency Weak Responding

9. Limits of Operant Conditioning Relevance Relevance Yawning to get food Yawning to get food Scratching a body part to get food Scratching a body part to get food Biting to get access to a female Biting to get access to a female Breland and “Misbehavior” Breland and “Misbehavior”

10. How to train a chicken

11. Schedules of Reinforcement You could give a reinforcement after each response You could give a reinforcement after each response This is called CRF or Continuous reinforcement This is called CRF or Continuous reinforcement Does not maintain high rates of behavior Does not maintain high rates of behavior

12. Schedules of Reinforcement Fixed Interval Fixed Interval First response after a given interval is rewarded First response after a given interval is rewarded FI Scallop FI Scallop Variable Interval Variable Interval Like FI but varies with a given average Like FI but varies with a given average Scallop disappears Scallop disappears

13. Schedules of Reinforcement Fixed Ratio Fixed Ratio Reinforcement is given after a given number of responses Reinforcement is given after a given number of responses Short pauses Short pauses Variable Ratio Variable Ratio After a varying number of responses After a varying number of responses

14. Some Other Schedules DRL, Differential reinforcement for low rates of responding DRL, Differential reinforcement for low rates of responding DRH, Differential reinforcement for high rates of responding DRH, Differential reinforcement for high rates of responding DR0, Different reinforcement of anything but the target behavior DR0, Different reinforcement of anything but the target behavior

15. Compound Schedules Different schedules are presented one-by-one, signaled (Multiple) or not (Mixed) by their own discriminative stimulus Different schedules are presented one-by-one, signaled (Multiple) or not (Mixed) by their own discriminative stimulus Reinforcement occurs after two or more schedules have been completed in succession with either discriminative stimuli (Chained) not (Tandem). Reinforcement occurs after two or more schedules have been completed in succession with either discriminative stimuli (Chained) not (Tandem). Two schedules are simultaneously in force (Concurrent), usually for different responses, and reinforcement on those schedules is independent of each other. Two schedules are simultaneously in force (Concurrent), usually for different responses, and reinforcement on those schedules is independent of each other.

16. FR-10FR-20 Schedule this…. Concurrent, Choice between two alternative schedules Concurrent, Choice between two alternative schedules change over delay (no “channel surfing”) change over delay (no “channel surfing”) AB VI-30VI-60

17. Matching Law B1/(B1+B2) = R1/(R1+R2) B1/(B1+B2) = R1/(R1+R2) B stands for numbers of a certain behavior B stands for numbers of a certain behavior R stands for numbers of a reinforcers earned R stands for numbers of a reinforcers earned

18. $5 today$50 wait$5 today$6 wait Schedule this…. Concurrent, Choice between an immediate small reward or a larger delayed reward Concurrent, Choice between an immediate small reward or a larger delayed reward AB

19. Self-Control…. Self-Control…. Concurrent Choice Concurrent Choice Human and nonhumans often chose a immediate small reward over a larger delayed reward (delayed rewards are “discounted”) Human and nonhumans often chose a immediate small reward over a larger delayed reward (delayed rewards are “discounted”)

20. Example of Impulsivity “Free” reinforcers given every 20s Lever press advances delivery of the first pellet, and deletes the second pellet So, if you press at 2 seconds, you get a pellet immediately, but you get no other pellets until the 60 second pellet is available. 20s40s60s

21. Delay of Reinforcement Delayed reinforcers are steeply discounted Delayed reinforcers are steeply discounted Loss of self-control and impulsivity Loss of self-control and impulsivity Reinforcer Potency Delay

22. Increasing Self-Control small LARGE AB Direct Choice (Concurrent)

23. small A LARGE B AB Concurrent Chain (Precommittment)

24. Self Control Behavioural Behavioural Precommitment Precommitment Self-Exclusion Contracts Self-Exclusion Contracts Distraction Distraction Modeling Modeling Shaping Waiting Shaping Waiting Reduce delay for small Reduce delay for small Increase delay for large Increase delay for large Cognitive Cognitive Public Declaration Public Declaration Abstinence Pledge Abstinence Pledge Abstinence Pledge Abstinence Pledge Cold vs Hot Thoughts Cold vs Hot Thoughts Increase Internal Resources for Self- Control Increase Internal Resources for Self- Control Counterfactual Learning Counterfactual Learning Dopamine Error Signals Dopamine Error Signals Experiential (actual) Experiential (actual) Fictive (could have) Fictive (could have)