1. The Matching Law Richard J. Herrnstein

2. Reinforcement schedule Fixed-Ratio (FR) : the first response made after a given number of responses is reinforced. (ex. Peicework system) Variable-Ratio (VR) : similar to FR except that the number of responses required varies between reinforcements. (ex. Gambling)

3. Reinforcement schedule Fixed-Interval (FI) : the first response made after a given time interval is reinforced. (ex. Stipend) Variable-Interval (VI) : similar to FI except the interval requirements vary between reinforcers around some specified average value. (ex. Fishing)

4. Ch1 – Experimental design In this experiment, the reinforcement were given to pigeons when they pecked on either of two keys. The reinforcement for one key was delivered on a variable-interval schedule which was independent of the schedule for the other key.

5. Ch1 - Experimental design The mean time interval between reinforcements on each key was the primary independent variable. These intervals were chosen so that the mean interval of reinforcement for the two keys was held constant at 1.5 minutes. For example, VI(3) VI(3); VI(2.25) VI(4.5); VI(1.5) VI(∞);

6. Ch1 – Result Relative frequency of responding to Key A. It is exactly equal to the relative frequency of reinforcement.

7. Ch1 – Result The absolute rate of responding on each of the keys. It is approximately a linear function that passes through the origin.

8. Ch1 – Discussion Rate of responding is a linear measure of response strength. The relative frequency of responding on a given key closely approximated the relative frequency of reinforcement on that key.

9. Ch4 – Maximizing vs Matching From a viewpoint of the maximizer, equilibrium is reached when a distribution of activities cannot improve the obtained outcomes by a redistribution of choices. Matching requires the ratio of the frequencies of any two behavior, B 1 and B 2, to match that of their obtained reinforcements, R 1 and R 2.

10. Ch4 - Melioration How does an organism come to match its distribution of choices to the obtained reinforcements? By shifting behavior toward higher local rates of reinforcement. If R D is zero, equilibrium is achieved. When R D >0, time allocation shifts toward t 1 ; when R D <0, it shifts toward t 2.

11. Ch4 – Comparison between matching and maximization Melioration implies that behavior maximizes total reinforcement, R T, under two and only two conditions, as follows:

12. Ch4 – Concurrent VR VR, where

13. Ch4 – Concurrent VR VR Melioration predicts preference for alternative 1, since R D >0 at all allocations. Maximization predicts likewise because R T is at its maximum at t 1 =1. When alternative 1 reinforces with a higher probability than alternative 2

14. Ch4 – Concurrent VI VR Local rate of reinforcement for VI, where V : scheduled average interreinforcement time d 1 : average interresponse time during responding on the VI I : a measure of interchangeover times between two schedules t 1 : proportion of time spent on the VI (1-t 1 ) : proportion of time on VR Local rate of reinforcement for VR

15. Ch4 – Concurrent VI VR, where,

16. Ch4 – Concurrent VI VR The optimal strategy for conc VI VR would seem to call for lots of time on the VR with occasional forays to the VI to collect a reinforcement come due. Nevertheless, no subject displayed any such bias toward VR. Solid : best-fitting line Dashed : prediction of matching Dot-dashed : prediction of maximization

17. Ch4 – Concurrent VI VR Divergence between two theories The value of R T when R D =0 is about 15 percent lower than when RT is maximized, which is the reinforcement cost of matching.

18. Ch4 – Mazur’s Experiment VI 45-second schedule, which randomly and equally often assigned each dark period to one key or the other. During the 3-second dark periods, a small ration of food was delivered with the following probability :

19. Ch4 – Mazur’s Experiment For a maximizer, the pigeons should always sample each alternative frequently and equally. For a matcher, the pigeons should shift preference along with the proportions of yielding food.

20. Ch4 – Mazur’s Experiment

22. Ch4 – Vaughan’s Experiment Modified Conc VI VI schedule Schedule values were updated every 4 minutes of responding. In condition a, the left schedule reinforces at a higher rate than the right schedule; in condition b, vice versa.

23. Ch4 – Vaughan’s Experiment Maximization picture In either condition, a subject earned the maximum, 180 reinforcements per hour, by spending 0.125-0.25 of its time responding to the right altenative.

24. Ch4 – Vaughan’s Experiment Melioration picture Melioration should have held choice within the interval from 0.125-0.25 during condition a and within the interval 0.75-0.875 during condition b.

25. Ch4 – Vaughan’s Experiment The results for the three pigeons

26. Limits of Melioration Comparative Most of data showing the different predict between maximizing and melioration came from the pigeons. Psychophysical Ambiguity in the meaning of “local” rates of reinforcement Motivational Food was used as the only reinforcement. Procedural Melioration can be generalized from concurrent to single- response procedures and multiple schedules, but there is no fully satisfactory formula for multiple-schedule responding yet. Inherent Limits Is the class of equally reinforced movements also the class of maximally reinforced movements?