1. Operant Conditioning Module 19
Module 19 edit

2. Learning Operant Conditioning Overview Skinner’s Experiments
Extending Skinner’s Understanding
Skinner’s Legacy
Contrasting Classical & Operant Conditioning
Module 19 edit

3. Edward L. Thorndike ( 1874–1949) DiscPsy Photo p179
Module 19 edit

4. Thorndike’s Puzzle Box link
Module 19 edit

5. Early Operant Conditioning
E. L. Thorndike (1898)
Puzzle boxes and cats
Hockenbury slides (Schulman)
Thorndike put cats into puzzle boxes and made them find the solution to their quandary.
Thorndike did not elicit a response as Pavlov had, he had to wait for the animal to emit the proper response, learn from it and do it again.
The trial and error process through which the animals learned the way to trip the latch was what Thorndike called his law of effect. Responses that produce a satisfying effect in a particular situation become more likely to occur again in that situation, and responses that produce a discomforting effect become less likely to occur again in that situation.
Instrumental responses- actions which function as tools to work some change in the environment; also called operant responses.
Ex: flipping a switch to light a room; rats pushing a lever to receive food
Operant conditioning- learning process by which the consequence of an operant response affects the likelihood that the response will occur in the future.
Scratch at bars
Push at ceiling
Dig at floor
Situation:
stimuli
inside of
puzzle box
Howl
Etc.
Press lever
First Trial
in Box
After Many
Trials in Box
Module 19 edit

6. DiscPsy Photo p179
B. F. Skinner (1904–1990)
Module 19 edit

7. B.F. Skinner and Operant Conditioning
B.F. Skinner had an enormous influence on psychology in general and on the field of psychology known as behaviorism in particular. His key theories were published in the early 1950s.
As Pavlov’s experiments showed, classical conditioning involves associating a neutral external stimulus with a response that is generally automatic (such as salivating). Skinner’s research revealed the power of operant conditioning, which involves learning how to operate on one’s environment to elicit a particular stimulus (a reward) or to avoid a punishment. In operant conditioning, the subject controls his or her response. You will learn how this works in the next few slides.
Operant Conditioning
type of learning in which behavior is strengthened if followed by reinforcement or diminished if followed by punishment
Classical conditioning involves an automatic response to a stimulus
Operant conditioning involves learning how to control one’s response to elicit a reward or avoid a punishment (to press a lever for example)
Module 19 edit

8. Skinner’s Experiments
Skinner’s experiments extend Thorndike’s thinking, especially his law of effect. This law states that rewarded behavior is likely to occur again.
Law of Effect
Thorndike’s principle that behaviors followed by favorable consequences become more likely, and behaviors followed by unfavorable consequences become less likely
Yale University Library
Module 19 edit

9. Operant Conditioning Operant Behavior Respondent Behavior
operates (acts) on environment
produces consequences
Respondent Behavior
occurs as an automatic response to stimulus
behavior learned through classical conditioning
Module 19 edit

10. Operant Chamber Skinner Box
chamber with a bar or key that an animal manipulates to obtain a food or water reinforcer
contains devices to record responses
developed behavioral technology
Module 19 edit

11. Operant Chamber Examples. Module 19 edit 10 20 11
Walter Dawn/ Photo Researchers, Inc.
From The Essentials of Conditioning and Learning, 3rd
Edition by Michael P. Domjan, Used with permission
by Thomson Learning, Wadsworth Division
Module 19 edit

12. The “Skinner Box” Rats placed in “Skinner boxes”
Skinner hypothesized that rats could be trained to perform specific behaviors in order to receive a food reward. He placed the rats into what is technically called an “operant chamber” but became more commonly known as a “Skinner box.” The soundproof glass box contained a bar or a key that the rat could press down to receive food. This bar or key was hooked up to an instrument that recorded how many times the rat pressed it.
Skinner used a process called “shaping” to teach the rats to press the bar for food. For example, if a rat approached the bar, he might initially give it a pellet of food as a reward for getting close to the bar. Skinner would gradually make the rat get closer to the bar before giving it food. Eventually, the rat learned that it had to press the bar in order to get any food.
The food in this case is referred to as a “reinforcer,” since it reinforces the rat’s behavior of stepping closer to and eventually pressing the bar.
Rats placed in “Skinner boxes”
Shaped to get closer and closer to the bar in order to receive food
Eventually required to press the bar to receive food
Food is a reinforcer
Module 19 edit

13. Shaping
Shaping is the operant conditioning procedure in which reinforcers guide behavior towards the desired target behavior through successive approximations. link
Shape behavior in class
The technique of strengthening behavior by reinforcing successive approximations is called (AP94)
(A) Positive reinforcement
(B) Negative reinforcement
(C) Distributed practice
(D) Modeling
(E) Shaping
Link Shaping a lever press in rats
Khamis Ramadhan/ Panapress/ Getty Images
Fred Bavendam/ Peter Arnold, Inc.
A rat shaped to sniff mines. A manatee shaped to discriminate
objects of different shapes, colors and sizes.
Module 19 edit

14. Shaping—an operant technique in which an organism is rewarded for closer and closer approximations of the desired response—is used in teaching both animals and humans. It is the main means of training animals to perform unnatural tricks. Breland and Breland’s (1961) famous subject, “Priscilla, the Fastidious Pig,” is shown above.
p. 228
Module 19 edit

15. Shaping—an operant technique in which an organism is rewarded for closer and closer approximations of the desired response—is used in teaching both animals and humans. It is the main means of training animals to perform unnatural tricks.
p. 228
Module 19 edit

16. Types of Reinforcers
Reinforcement: Any event that strengthens the behavior it follows.
A heat lamp positively reinforces a meerkat’s behavior in the cold.
Preview Question 7: What are the basic types of reinforcers?
Taking a painkiller to relieve a toothache is behavior learned through which of the following processes? (AP99)
(A) Shaping
(B) Punishment
(C) Positive reinforcement
(D) Negative reinforcement
(E) Omission training
Reuters/ Corbis
Module 19 edit

17. Types of Reinforcement
Positive reinforcer (+)
Adds something rewarding following a behavior, making that behavior more likely to occur again
Giving a dog a treat for fetching a ball is an example
Negative reinforcer (-)
Removes something unpleasant that was already in the environment following a behavior, making that behavior more likely to occur again
Taking an aspirin to relieve a headache is an example
Golf claps
Low grade electrical shock only removed when you performed the task
Module 19 edit

18. Module 19 edit

19. Module 19 edit

20. Escape and Avoidance: Two types of negative reinforcement
Removal of an aversive stimulus by doing a behavior
Escape Conditioning
Avoidance Conditioning
Adapted from: The Psychology of Memory and Learning by Hintzman. © 1978 by W.H. Freeman and Company. Used with permission.
Module 19 edit

21. Learned Helplessness
Failure to try to avoid an unpleasant stimulus because in the past it was unavoidable
Possible model for depression in humans
Module 19 edit

22. Kinds of Reinforcement and Punishment
Positive +
(adding stimulus)
Negative –
(removing stimulus)
Reinforcement
(label afterwards to describe increase in behavior)
Punishment
(label afterwards to describe decrease in behavior)
Module 19 edit

23. Kinds of Reinforcement and Punishment
Positive +
(adding stimulus)
Negative –
(removing stimulus)
Reinforcement
(label afterwards to describe increase in behavior)
Pos. Reinf.
(Adding pleasant consequence)
Punishment
(label afterwards to describe decrease in behavior)
Module 19 edit

24. Kinds of Reinforcement and Punishment
Positive +
(adding stimulus)
Negative –
(removing stimulus)
Reinforcement
(label afterwards to describe increase in behavior)
Pos. Reinf.
(Adding pleasant consequence)
Neg. Reinf.
(Removing Aversive Stimuli)
Punishment
(label afterwards to describe decrease in behavior)
Module 19 edit

25. Kinds of Reinforcement and Punishment
Positive +
(adding stimulus)
Negative –
(removing stimulus)
Reinforcement
(label afterwards to describe increase in behavior)
Pos. Reinf.
(Adding pleasant consequence)
Neg. Reinf.
(Removing Aversive Stimuli)
Punishment
(label afterwards to describe decrease in behavior)
Pos. Pun.
(Adding aversive stimuli)
Module 19 edit

26. Kinds of Reinforcement and Punishment
Positive +
(adding stimulus)
Negative –
(removing stimulus)
Reinforcement
(label afterwards to describe increase in behavior)
Pos. Reinf.
(Adding pleasant consequence)
Neg. Reinf.
(Removing Aversive Stimuli)
Punishment
(label afterwards to describe decrease in behavior)
Pos. Pun.
(Adding aversive stimuli)
Neg. Pun.
(Removing pleasant stimuli)
Module 19 edit

27. Examples Link 1 Link 2 Link
Link – reinforcement in the service 2
Link 1
Link 2
Module 19 edit

28. Negative Reinforcement and Punishment
Negative reinforcement: Removing an unpleasant stimulus
Punishment
In operant conditioning, negative reinforcement and punishment are two different things that can be easy to confuse.
The food rewards Skinner used are known as positive reinforcement. Negative reinforcement, on the other hand, occurs when an unpleasant stimulus is removed. For example, in one experiment Skinner would play a loud noise inside the box; to make the noise stop, the rat would have to press the bar.
Punishment involves either the introduction of an unpleasant stimulus or the removal of a pleasant stimulus. When Skinner gave a rat a painful electric shock after pushing the bar, the rat learned not to push the bar. Another example of punishment would be a parent withholding dessert from a child who misbehaves at the dinner table.
1. Introducing an unpleasant stimulus
1. Unpleasant stimulus =
2. Removal of unpleasant stimulus
2. Withholding a pleasant stimulus =
Module 19 edit

29. Module 19 edit

30. Figure 6.18 Positive reinforcement versus negative reinforcement
Module 19 edit

31. Figure 6.20 Comparison of negative reinforcement and punishment
Module 19 edit

32. IMPORTANT!! Negative reinforcement is NOT punishment.
Negative reinforcement is the REMOVAL of unpleasant stimulus when target behavior is observed (a positive consequence of behavior – increases behavior)
Punishment is the introduction of an aversive (unpleasant) stimulus or removal of a pleasant stimulus as a consequence of behavior – ( a negative consequence of behavior - decreases behavior.
Although there may be some justification for occasional punishment (Larzelaere & Baumrind, 2002), it usually leads to negative effects.
Results in unwanted fears.
Conveys no information to the organism.
Justifies pain to others.
Causes unwanted behaviors to reappear in its absence.
Causes aggression towards the agent.
Causes one unwanted behavior to appear in place of another.
Module 19 edit

33. An aversive event that decreases the behavior it follows.
Punishment
An aversive event that decreases the behavior it follows.
Preview Question 9: How does punishment affect behavior?
If a man who is a heavy smoker is given an electric shock every time he takes a puff on a cigarette, which of the following behavior-modification techniques is being used? (AP94)
(A) Systematic desensitization
(B) Modeling
(C) Aversive conditioning
(D) Homogeneous reinforcement
(E) lnterlocking reinforcement
Punishment is most effective in eliminating undesired behavior when the (AP99)
(A) Behavior is complex
(B) Behavior was very recently acquired
(C) Punishment is delivered soon after the behavior
(D) Punishment is delivered by someone with authority
(E) Punishment is both mental and physical
Module 19 edit

34. Primary & Secondary Reinforcers
Primary Reinforcer: An innately reinforcing stimulus like food or drink. (satisfies a biological need
Conditioned (secondary) Reinforcer: A learned reinforcer that gets its reinforcing power through association with the primary reinforcer.
PR satisfies a biological need
Secondary – Money, good grades, pleasant tones of voice
Module 19 edit

35. Immediate & Delayed Reinforcers
Immediate Reinforcer: A reinforcer that occurs instantly after a behavior. A rat gets a food pellet for a bar press.
Delayed Reinforcer: A reinforcer that is delayed in time for a certain behavior. A paycheck that comes at the end of a week.
We may be inclined to engage in small immediate reinforcers (watching TV) rather than large delayed reinforcers (getting an A in a course) which require consistent study.
Module 19 edit

36. Reinforcement Schedules
Continuous Reinforcement: Reinforces the desired response each time it occurs.
Partial (intermittent) Reinforcement: Reinforces a response only part of the time. Though this results in slower acquisition in the beginning, it shows greater resistance to extinction later on.
Preview Question 8: How do different reinforcement schedules affect behavior?
2 pigeon may peck 150,000 times without reward (Skinner 1953)
Module 19 edit

37. Schedules of Reinforcement
Partial reinforcement lies between continuous reinforcement and extinction
Module 19 edit

38. Schedules of Reinforcement
Fixed Ratio (FR)
reinforces a response only after a specified number of responses
faster you respond the more rewards you get
different ratios
very high rate of responding
like piecework pay
Module 19 edit

39. Schedules of Reinforcement
Variable Ratio (VR)
reinforces a response after an unpredictable number of responses
like gambling, fishing
very hard to extinguish because of unpredictability
Skinner link 3:58
Reinforcement schedules: variable ratio
SLOT machines show SLOwesT extinction.
Module 19 edit

40. Schedules of Reinforcement
Fixed Interval (FI)
reinforces a response only after a specified (fixed) time has elapsed
response occurs more frequently as the anticipated time for reward draws near
Notice that responding increases right before the end of the interval period.
Timing of Reinforcer – Usually the shorter the delay between behavior and reinforcement, the more effective.
Size of Reinforcer – Usually the larger the reinforcer, the more effective. (think lotto)
Module 19 edit

41. Schedules of Reinforcement
Variable Interval (VI)
reinforces a response at unpredictable time intervals
produces slow steady responding
like pop quiz
Module 19 edit

42. Intermittent Reinforcement Schedules Summary
Based on Number of necessary responses
Based on Time that must first pass
Fixed Ratio
(FR)
Fixed Interval
(FI)
Predictable
Variable Ratio
(VR)
Variable Interval
(VI)
Unpredictable (“On the Average”)
Module 19 edit

43. Schedules of Reinforcement
Response patterns for pigeons
Module 19 edit

44. You do not have to write down the following examples.
Module 19 edit

45. FI, VI, FR, or VR?
When I bake cookies, I can only put one set in at a time, so after 10 minutes my first set of cookies is done. After another ten minutes, my second set of cookies is done. I get to eat a cookie after each set is done baking.
After every 10 math problems that I complete, I allow myself a 5 minute break.
I look over my notes every night because I never know how much time will go by before my next pop quiz.
When hunting season comes around, sometimes I’ll spend all day sitting in the woods waiting to get a shot at a big buck. It’s worth it though when I get a nice 10 point.
Today in Psychology class we were talking about Schedules of Reinforcement and everyone was eagerly raising their hands and participating. Miranda raised her hand a couple of times and was eventually called on. FI FR VI VR
Module 19 edit

46. FI, VI, FR, or VR? 6. FR 7. FI 8. VR VI FI FR
6. Madison spanks her son if she has to ask him three times to clean up his room.
7. Emily has a spelling test every Friday. She usually does well and gets a star sticker.
8. Steve’s a big gambling man. He plays the slot machines all day hoping for a big win.
Snakes get hungry at certain times of the day. They might watch any number of prey go by before they decide to strike.
Mr. Bertani receives a salary paycheck every 2 weeks. (Miss Suter doesn’t ).
Christina works at a tanning salon. For every 2 bottles of lotion she sells, she gets 1 dollar in commission.
Mike is trying to study for his upcoming Psychology quiz. He reads five pages, then takes a break. He resumes reading and takes another break after he has completed 5 more pages.
6. FR
7. FI
8. VR VI FI FR
Module 19 edit

47. FI, VI, FR, or VR? 13. VR 14. VI 15. VI 16. FI 17. FR 18. VR
13. Megan is fundraising to try to raise money so she can go on the annual band trip. She goes door to door in her neighborhood trying to sell popcorn tins. She eventually sells some.
14. Kylie is a business girl who works in the big city. Her boss is busy, so he only checks her work periodically.
15. Mark is a lawyer who owns his own practice. His customers makes payments at irregular times.
16. Jessica is a dental assistant and gets a raise every year at the same time and never in between.
17. Andrew works at a GM factory and is in charge of attaching 3 parts. After he gets his parts attached, he gets some free time before the next car moves down the line.
18. Brittany is a telemarketer trying to sell life insurance. After so many calls, someone will eventually buy.
13. VR
14. VI
15. VI
16. FI
17. FR
18. VR
Module 19 edit

48. Updating Skinner’s Understanding
Skinner’s emphasis on external control of behavior made him an influential, but controversial figure.
Many psychologists criticized Skinner for underestimating the importance of cognitive and biological constraints.
19 7. One major objection to the early Skinnerian approach to psychology is that it (AP99)
(A) Did not take into account internal thoughts and feelings
(B) Did not take into account overt physical behaviors
(C) Did not take into account accumulated experiences
(D) Focused primarily on childhood experiences
(E) Focused primarily on the unconscious
Module 19 edit

49. Cognitive Approach
This approach emphasizes abstract and subtle learning that could not be achieved through conditioning or social learning alone.
Expectancies are beliefs about our ability to perform an action and to get the desired reward. Expectancies affect learning.
Module 19 edit

50. Cognition & Operant Conditioning
Evidence of cognitive processes during operant learning comes from rats during a maze exploration in which they navigate the maze without an obvious reward. Rats seem to develop cognitive maps (E.C. Tolman), or mental representations, of the layout of the maze (environment).
OBJECTIVE 16| Explain how latent learning and the effect of external rewards demonstrate that cognitive processing is an important part of learning
Cognition is happening
Allow rats to explore a maze without reward, then add reward, rats run maze very quickly, seem to have mental maps of maze, even without reinforcement….
This is evidence of cognition
Module 19 edit

51. Latent Learning Latent learning
Learning that only becomes apparent when there is some incentive to demonstrate it.
Some learning is not intentional, but occurs almost accidentally—a situation called latent learning. Learning that occurs, but is not apparent until there is an incentive to demonstrate it
Module 19 edit

52. Intrinsic Motivation
Intrinsic Motivation: The desire to perform a behavior for its own sake.
Extrinsic Motivation: The desire to perform a behavior due to promised rewards or threats of punishments.
Kids given interesting toy and promised a reward later play with toy less than kids not promised a reward (Deci 1999)
Excessive rewards undermine intrinsic motivation….did your grade school teachers give you a treat every time you blinked?
Module 19 edit

53. Biological Predisposition
Biological constraints predispose organisms to learn associations that are naturally adaptive.
Breland and Breland (1961) showed that animals drift towards their biologically predisposed instinctive behaviors.
Can condition normal behaviors
Pigeon flap-to avoid a shock (escape behavior)
Peck-food
Not opposite
Hamster can be taught to dig or rear up for food but not face washing, because digging and rearing up are food gathering behaviors
Photo: Bob Bailey
Marian Breland Bailey
Module 19 edit

54. Falk/ Photo Researchers, Inc.
Skinner’s Legacy
Skinner argued that behaviors were shaped by external influences instead of inner thoughts and feelings. Critics argued that Skinner dehumanized people by neglecting their free will.
Falk/ Photo Researchers, Inc.
Module 19 edit

55. Applications of Operant Conditioning
Skinner introduced the concept of teaching machines that shape learning in small steps and provide reinforcements for correct rewards.
Preview Question 11: How might educators, business managers, and other individuals apply operant conditioning?
LWA-JDL/ Corbis
In School
Module 19 edit

56. Applications of Operant Conditioning
Reinforcers affect productivity. Many companies now allow employees to share profits and participate in company ownership.
At work
Module 19 edit

57. Applications of Operant Conditioning
At Home
In children, reinforcing good behavior increases the occurrence of these behaviors.
Ignoring unwanted behaviors decreases their occurrence.
Module 19 edit

58. EXPLORING PSYCHOLOGY (7th Edition in Modules) David Myers
PowerPoint Slides
Aneeq Ahmad
Henderson State University, Amy Jones, Bernstein, Schallhorn with Garber edits
Worth Publishers, © 2008
Module 19 edit