1. Chapter 9 Adjusting to Schedules of Partial Reinforcement

2. Schedules of Reinforcement Specify the conditions the individual has to satisfy in order to obtain the reinforcing event These conditions can include… How many times the individual must perform the behavior When the behavior has to be performed Performing a certain pattern of behaviors

3. Classifying Schedules of Reinforcement Continuous Reinforcement = provides the consequence for each occurrence of the target behavior Schedule of Partial (Intermittent) Reinforcement = specifies which instances of the target behavior are followed by the consequence It is the type of schedule established when not every instance of the target behavior is followed by the consequence Most of these schedules require that the individual either perform the target behavior a certain number of times, perform that behavior after a certain time interval has elapsed or some combo of both e.g., The Lottery takes advantage of the fact that some individuals will continue to play even though they win only occasionally

4. 3 Most Common Classes of Schedules 1) Ratio Schedules = based on the number of individual actions required to obtain a reinforcer 2) Interval Schedules = behaving after a certain time interval has elapsed in order to obtain a reinforcer 3) Interresponse Time Schedules = spacing individual actions a certain time apart in order to be rewarded

5. Ratio Schedules Specify the number of times an individual has to do something in order to obtain a certain consequence That is the only requirement The consequence occurs each time the requirement is satisfied Direct Variables = formally imposed requirements of a schedule Indirect Variables = unstated requirements These variables affect how the individual adjusts to it e.g., in all ratio schedules, the faster the individual gets through the ratio, the faster he or she earns the consequence

6. Ratio-Schedules Cont. Ratio schedules tie the rate of reinforcement directly to the rate of behavior Molar Feedback Functions = define the relationship b/w the rate of performance of the target behavior and the rate at which the consequence is earned These are called molar functions b/c they relate the overall rate of reinforcement to the overall rate of behavior Molar feedback functions for ratio schedules encourage high rates of behavior b/c pausing & dawdling delay earning the next reward Molecular Feedback = involves the moment-to- moment relationship b/w behaviors and consequences

7. Ratio Schedules Cont. Ratio schedules are more likely to reinforce short interresponse times (IRTs) rather than long ones b/c the IRT that is followed immediately by reinforcement is likely to be a short one, and short IRTs are synonymous with rapid behavior Most common types of ration schedules: 1) Fixed Ratio 2) Variable Ratio

8. Fixed-Ratio Schedules (FR n) The number of times the target behavior must occur is the same across successive reinforcers Symbolized as FR n where n specifies the required number of times the target behavior must occur in order to obtain the consequence e.g., An FR 5 schedule means that the individual must always perform the required action 5 times in order to obtain a certain consequence Continuous Reinforcement – is a special case of a fixed-ratio schedule in the which the ratio requirement is 1 (FR 1)

9. Typical Performance on Fixed- Ratio Schedules “Break and Run” Pattern: the “break” refers to a pause for a period of time after the presentation of each consequence The length of this pause called a Post- Reinforcement Pause (PRP) is typically an increasing function of the fixed-ratio requirement: the longer the required ratio, the longer the pause Following the PRP (the “break”), the individual typically makes an abrupt transition into the “run,” a high steady rate of behavior that ends with the delivery of reinforcement

10. Why there is a post-reinforcement pause on fixed-ratio schedules The time b/w the receipt of successive reinforcers called the Interreinforcer Interval depends on how long the individual takes to complete each action and the number of actions required by the schedule This interval affects the length of the post- reinforcement pause (PRP): the longer the interreinforcer interval, the longer the PRP

11. Examples of Fixed-Ratio Schedules in Everyday Life Working on Commission, being paid for the number of tasks or objects completed (piece work), or receiving a bonus for each item or task above the minimum number In all these 3 cases, the amount of pay earned depends on the number of items sold (or made) based on a pay schedule which specifies how much is earned per sale or item produced

12. Variable-Ratio Schedules (VR n) Require the individual to perform the target behavior a certain number of times to obtain the consequence, but that number varies around a certain average (n) VR schedules generate high rates of behavior e.g., a VR 5 schedule means that the individual will obtain the consequence after performing the target behavior an average of 5 times such that sometimes a single instance is sufficient, sometimes 3, sometimes 10, and other times perhaps 15 times

13. Examples of Variable-Ratio Schedules in Everyday Life All forms of Gambling – winning depends on betting, and the more times you bet, the more likely you are to win In Basketball, not every shot is successful, the more you shoot, the more likely you are to get a score These examples are different from other schedules b/c the individual continues to work on the schedule after receiving the reward; in all the other schedules, once the reward is earned, the individuals typically go on to perform other tasks

14. FI vs. FR Schedule

15. Interval Schedules Require the individual to perform the target behavior after a certain time has elapsed; it does not matter what the individual does before that time Note: it is not merely the passage of time that produces the consequence – the individual has to do something after that interval has elapsed in order for the consequence to occur As long as the individual is performing the target behavior some minimum number of times over the course of the experimental session, he/she will earn the maximum number of reinforcers possible, but more rapid behavior will not earn any more. Therefore, unlike ratio schedules, after a certain point, the rate of reinforcement is independent of the rate of behavior

16. Interval Schedules Cont. These schedules encourage lower rates of behavior than do ratio schedules b/c pausing and low rates of behavior do not appreciably delay the consequence The most common ways to program the interval schedule requirement are: 1) To require the same interval for each reinforcer (Fixed Interval) 2) To unsystematically vary the interval across successive possibilities (Variable Interval)

17. Fixed-Interval Schedule (FI t) The first occurrence of the target behavior after a specified interval of time has elapsed will produce the consequence; that interval is the same across successive reinforcers Symbolized as FI t, where t specifies the time interval in whatever units are appropriate (seconds, minutes, and so on) e.g., an FI 1 minute schedule programs the consequence following the first instance of the target behavior that occurs after 1 minute has elapsed since the last consequence

18. Typical Performance on Fixed- Interval Schedules Performance on FI schedules is characterized by a post-reinforcement pause followed by either a progressively increasing rate of behavior or an abrupt transition to the terminal rate of behavior (break and run) These schedules usually generate higher rates of behavior closer to the time of reinforcement e.g., delaying studying until just before a test e.g., waiting for something to cook (You are rewarded for checking your food when it is done; if you check it before it is done, you have wasted some effort)

19. Variable-Interval Schedules (VI t) Requires the individual to perform the target behavior after a certain time has elapsed, but that time is not the same for each reinforcer These schedules generate slow, steady rates of behavior with little or no post-reinforcement pauses e.g., a VI 1 minute schedule, the individual can obtain the consequence for the first occurrence of the target behavior after a certain time has elapsed, but the required time will vary across successive reinforcers with an average interreinforcer time of 1 minute

20. Everyday Examples of VI Schedules If you are trying to call a friend on the telephone and the line is busy (b/c that person does not have call waiting), you tend to wait for some time and then try again. Repeatedly pressing the redial button does not speed up your getting through, but if you wait too long, that person may hang up and take another call

21. VI vs VR Schedule

22. Schedules of Reinforcement

23. The Effects of Prior Experience How an individual adjusts to a schedule of partial reinforcement is usually affected by his or her prior experiences Individuals working under one schedule of reinforcement and then shifted to a second schedule may perform differently on the second schedule than individuals who did not have that prior experience with the first schedule Repeated experience with change makes individuals more sensitive to change so that they more readily adapt to it These carryover effects provide another perspective on how individuals adjust to changing circumstances

24. Behavioral Persistence The tendency to continue to perform a behavior when that behavior no longer produces a reinforcing event The conventional measure is called Resistance to Extinction how long it takes an individual to cease performing a behavior when that behavior no longer produces the reinforcing event (the procedure of extinction) the longer it takes the individual to cease performing a behavior when that behavior no longer produces the reinforcing event, the greater the resistance to extinction

25. 3 Factors that Influence Behavioral Persistence 1) How much success the individual has experienced obtaining the reinforcing event by performing the target behavior: the greater the amount of success, the more persistent the behavior 2) How discriminable the change in procedure is to the individual - i.e., how quickly the individual can detect that reinforcement is no longer available Discriminability is determined by how similar the 2 situations are: with respect to persistence, the more similar (less discriminable) extinction is to training, the greater the persistence (the more resistance to extinction) 3) Having had prior experience in which persistence has been rewarded: experience with partial reinforcement produces resistance to extinction

26. Behavioral Momentum The tendency for a behavior to persist when the reinforcement contingency is changed Behaviors that are more resistant to change have more momentum From this perspective, amount of training increases persistence (resistance to extinction) b/c successful behaviors acquire more momentum and thus become more resistant to change when extinction is introduced

27. Discriminability and Persistence Persistence is a function of how discriminable the conditions of reinforcement are from the conditions of extinction: extinction occurs faster (there is less persistence) when these two are more easily discriminated Discriminability is related to Salience and is determined in part by the physical attributes of the stimulus and in part by what the individual brings to the situation (sensory capacities, knowledge) Two events are discriminable when the individual can tell them apart

28. Persistence and Partial Reinforcement Behaviors maintained by partial reinforcement persist longer in extinction than behaviors maintained by continuous reinforcement This is called the Partial Reinforcement Extinction Effect (PREE) The PREE is due in part to the fact that it is easier for individuals to discriminate the difference between continuous reinforcement and extinction than b/w partial reinforcement and extinction, and in part to the fact that experience with partial reinforcement teaches individuals to tolerate frustration

29. Persistence & Partial Reinforcement Cont. Experience with partial reinforcement increases persistence: individuals who experience partial reinforcement, even if their most recent experience is with continuous reinforcement, show more resistance to extinction (persistence) than individuals who only experience continuous reinforcement of that behavior This occurs because partial reinforcement increases frustration tolerance