1. What is difference b/w classical and operant? HW:
Aim: How are classical and Operant conditioning differed/ similar? How do we learn in Operant Conditioning?
Do Now:
What is difference b/w classical and operant?
HW:
Choose a behavior for Behavior Modification Project
Learning Test – 1/3

2. Operant vs. Classical Conditioning
If the organism is learning associations between its behavior and the resulting events, it is...
Operant vs. Classical Conditioning
operant conditioning
If the organism is learning associations between events that it does not control, it is...
Automatic animation for “operant conditioning” question sequence.
Click to reveal “classical conditioning” question sequence.
classical conditioning

3. Classical vs Operant Conditioning
Classical Conditioning
Operant Conditioning
Behavior is determined by what PRECEDES it.
Behavior is determined by the anticipation of what FOLLOWS it.
Involuntary
Voluntary
Dog salivates after a tone
Dog sits in anticipation of getting a treat.

4. Which is which?
Page 4 – Classical or Operant???

5. The Law of Effect Edward Thorndike Locked cats in a cage
Behavior changes because of its consequences.
Rewards strengthen behavior.
If consequences are unpleasant, the Stimulus-Reward connection will weaken.
Called the whole process instrumental learning.

6. Organism operates on the environment to acquire some consequences.
Operant Conditioning
Learning in which behavior is strengthened by reinforcement or weakened by punishment.
Organism operates on the environment to acquire some consequences.
Builds on Thorndike’s principle of Law of Effect: rewarded behavior is likely to recur .

7. Operant Conditioning
How it works:
An act of chosen behavior (a “response”) is followed by a reward or punitive feedback from the environment.
Results:
Reinforced behavior is more likely to be tried again.
Punished behavior is less likely to be chosen in the future.
Operant conditioning involves adjusting to the consequences of our behaviors. Examples:
We may smile more at work after this repeatedly gets us bigger tips.
We learn how to ride a bike using the strategies that don’t make us crash.
Click to reveal bullets and example.
Instructor: the word “strengthened” is used here to refer to making the behavior happen more frequently, more intensely, or in the case of the seals, more accurately and reliably.
Response: balancing a ball
Consequence: receiving food
Behavior strengthened

8. BF Skinner Father of operant conditioning techniques.
Skinner Box (Operant Chamber): major tool used in operant conditioning research.
“Box” contains a bar or key that an animal can manipulate to obtain reinforcement.

9. Skinner Box

10. Shaping
How would you have trained this cat to become potty trained?
An operant conditioning technique in which you mold a behavior, by reinforcing the organism as it gets closer and closer towards a desired goal ( successive approximations)

11. Shaping Behavior Reinforcing Successive Approximations
Shaping: guiding a creature toward the behavior by reward behavior that comes closer and closer to the desired behavior.
Shaping the teacher: Students could smile and nod more when the instructor moves left, until the instructor stays pinned to the left wall.
No animation.
Instructor: the “shape the instructor” exercise carries a myth that it was done in one class until the professor went out the door.
In a more realistic example, animal training makes use of shaping by rewarding successive approximations. (an illustration with seals occurs elsewhere in this slide show). A seal may not naturally balance a ball on its nose for a long time, but it might nudge it up into the air. By rewarding this behavior, the behavior can get stronger and more prolonged and can be rewarded when done after a hand gesture so that the gesture can serve as a command.

12. Behavior Modification Assignment
Pick a Behavior
Observe and record baseline (4 days) – frequency or duration
Design Behavior Modification plan using classical or operant
Implement the plan – 4 days
Create a histogram
2 – 3 page write up (target behavior, baseline data, plan, results, histogram) Due 1/9

13. Aim: How can reinforcement and punishments be used to shape behavior?
Do Now/Warm Up:
Go over test.
*What behavior will you observe/track? How will you measure it?
**Imagine you had to train a tiger to jump thru a flaming hoop. Apply the basic principle of shaping to achieving this goal.
HW:
Complete p. 15 – 17 in packet.
Day 1 Behavior Modification Tomorrow

14. Shaping a Tiger Shaping Successive Approximations
With Acquisition use Continuous Reinforcement – so the stimuli response connection is strengthened!

15. Chaining Behaviors
Subjects are taught a number of responses successively in order to get a reward.
Click picture to see a rat chaining behaviors.
Click to see a cool example of chaining behaviors.

16. Reinforcement
Reinforcement: feedback from the environment that makes a behavior more likely to be done again.
Positive + reinforcement: the reward is adding something desirable
Negative - reinforcement: the reward is ending something unpleasant
This meerkat has just completed a task out in the cold
For the meerkat, this warm light is desirable.
Click to reveal bullets.
You can give students a preview of upcoming concepts by pointing out that taking the warm light away from the meerkat could be used as a punishment (a negative punishment, because it’s taking something away).

17. Operant Effect: Punishment
Punishments have the opposite effects of reinforcement. These consequences make the target behavior less likely to occur in the future.
+ Positive Punishment
You ADD something unpleasant/aversive (ex: spank the child)
- Negative Punishment
You TAKE AWAY something pleasant/ desired (ex: no TV time, no attention)--MINUS is the “negative” here
Click to reveal bullets.
Positive does not mean “good” or “desirable” and negative does not mean “bad” or “undesirable.”

18. Summary: Types of Consequences
Adding stimuli
Subtract stimuli
Outcome
Positive + Reinforcement
(You get candy)
Negative –Reinforcement
(I stop yelling)
Strengthens target behavior
(You do chores)
Positive + Punishment
(You get spanked)
Negative –Punishment
(No cell phone)
Reduces target behavior
(cursing)
No animation.
uses desirable stimuli
uses unpleasant stimuli

19. Summary: Types of Consequences
Adding stimuli
Subtract stimuli
Outcome
Positive + Reinforcement
(You get candy)
Negative –Reinforcement
(I stop yelling)
Strengthens target behavior
(You take notes in class)
Positive + Punishment
(You get spanked)
Negative –Punishment
(No cell phone)
No animation.
uses desirable stimuli
uses unpleasant stimuli

20. Types of Consequences Reinforcement versus Punishment p. 17
Increases the likelihood of a behavior
Punishment:
Decreases the likelihood of a behavior
Positive: adding something
Negative: taking something away

21. Having a headache and taking an aspirin.
Positive or Negative?
Putting your seatbelt on.
Having a headache and taking an aspirin.
Studying for a test.
Getting a kiss for doing the dishes.
Faking sick to avoid AP Psych class.
Breaking out of jail.

22. Punishment Meant to decrease a behavior. Positive Punishment
Addition of something unpleasant.
Negative Punishment (Omission Training)
Removal of something pleasant.
Punishment works best when it is immediately done after behavior and if it is harsh!

23. Types of Reinforcers
Primary Reinforcer: an innately reinforcing stimulus, such as one that satisfies a biological need.
Ex: Food, sex.
Secondary Reinforcer (Conditioned Reinforcer): reinforcer that you have to learn usually through its association with a primary reinforcer.
Ex: Money, Applause, lever in skinner box.

24. 10 Super Cool Dog Tricks

25. Take out your HW Packet page 15 and 16
Aim: How can schedules of reinforcement be used to shape behavior? What are some limitations of operant conditioning?
Do Now:
New Seats
Take out your HW Packet page 15 and 16
What is the difference b/w primary and secondary reinforcers? Intrinsic and extrinsic reinforcers?
HW:
Day 1 – Baseline
Complete p. 25 in packet – Schedules of Reinforcement

26. A cycle of mutual reinforcement
Children who have a temper tantrum when they are frustrated may get positively reinforced for this behavior when parents occasionally respond by giving in to a child’s demands.
Result: stronger, more frequent tantrums
Parents who occasionally give in to tantrums may get negatively reinforced when the child responds by ending the tantrum.
Result: parents giving-in behavior is strengthened (giving in sooner and more often)
Click to reveal bullets.

27. What behavior is Sheldon molding. What reinforcements is he applying
What behavior is Sheldon molding? What reinforcements is he applying? What schedule of reinforcement is being applied?

28. A Human Talent: Responding to Delayed Reinforcers
Dogs learn from immediate reinforcement; a treat five minutes after a trick won’t reinforce the trick.
Humans have the ability to link a consequence to a behavior even if they aren’t linked sequentially in time.
A piece of paper (paycheck) can be a delayed reinforcer, paid a month later, if we link it to our performance.
Delaying gratification, a skill related to impulse control, enables longer-term goal setting.
Click to reveal bullets.

29. How often should we reinforce?
Do we need to give a reward every single time? Or is that even best?
B.F. Skinner “schedules” to determine what worked best to establish and maintain a target behavior.
In continuous reinforcement (giving a reward after the target every single time), the subject acquires the desired behavior quickly.
In partial/intermittent reinforcement (giving rewards part of the time), the target behavior takes longer to be acquired/established but persists longer without reward.
Click to reveal bullets.
A pigeon given a reward sometimes for pecking a key, later pecked 150,000 times without getting a reward. Trying to think like this pigeon: if the reward sometimes comes even after not showing up for a while, maybe the 150,001st time will be the time I get the reward.
Perhaps this is why neglected kids hold out hope of parental love and keep trying for it even after years without a response; maybe on some rare occasion, there was an apparently loving moment.

30. Reinforcement Schedules
How often to you give the reinforcer?
Every time or just sometimes you see the behavior.

31. Different Schedules of Partial/Intermittent Reinforcement p. 23 - 24
We may schedule our reinforcements based on an interval of time that has gone by.
Fixed interval schedule:
Every so often
Variable interval schedule:
Unpredictably often
We may plan for a certain ratio of rewards per number of instances of the desired behavior.
Fixed ratio schedule:
Every so many behaviors
Variable ratio schedule:
After an unpredictable number of behaviors
Click to reveal two text boxes.
Examples of each of the above:
Fixed interval schedule--paid once a week
Variable interval schedule--patted on the back occasionally
Fixed ratio schedule--paid for every 10 units of work completed
Variable ratio schedule--unpredictably getting a “good job” comment about some unit of work you completed

32. Continuous v. Partial Reinforcement
Reinforce the behavior EVERYTIME the behavior is exhibited.
Usually done when the subject is first learning to make the association.
Acquisition comes really fast.
But so does extinction.
Reinforce the behavior only SOME of the times it is exhibited.
Acquisition comes more slowly.
But is more resistant to extinction.
FOUR types of Partial Reinforcement schedules.

33. Reinforcement Schedule Patterns

34. Rapid responding near time for reinforcement
Results of the different schedules of reinforcement Which reinforcements produce more “responding” (more target behavior)?
Rapid responding near time for reinforcement
Fixed interval
Fixed interval: slow, unsustained responding
If I’m only paid for my Saturday work, I’m not going to work as hard on the other days.
Variable interval: slow, consistent responding
If I never know which day my lucky lottery number will pay off, I better play it every day.
Steady responding
Variable interval
Click to reveal bullets.

35. Effectiveness of the ratio schedules of Reinforcement
Fixed ratio
Fixed ratio: high rate of responding
Buy two drinks, get one free? I’ll buy a lot of them!
Variable ratio: high, consistent responding, even if reinforcement stops (resists extinction)
If the slot machine sometimes pays, I’ll pull the lever as many times as possible because it may pay this time!
Reinforcers
Variable ratio
Click to reveal bullets.

36. Which Schedule of Reinforcement is This. Ratio or Interval
Which Schedule of Reinforcement is This? Ratio or Interval? Fixed or Variable?
Rat gets food every third time it presses the lever
Getting paid weekly no matter how much work is done
Getting paid for every ten boxes you make
Hitting a jackpot sometimes on the slot machine
Winning sometimes on the lottery you play once a day
Checking cell phone all day; sometimes getting a text
Buy eight pizzas, get the next one free
Fundraiser averages one donation for every eight houses visited
Kid has tantrum, parents sometimes give in
Repeatedly checking mail until paycheck arrives FR FI VR VI
Click to reveal question, then answer.
Instructor: on the first few items, or for students who may struggle, you could call on people to simply state whether it’s ratio or interval, or whether it’s fixed or variable. However, after #5, ask for answers stating the whole term such as “fixed interval.”
Explanations for selected items:
#5. If you play once a day, then the ratio (percentage of behaviors rewarded) and the interval (percentage of days rewarded) are the same, so both answers are correct.
#6. The reward for checking for messages is a function of time, not a function of how often you check. This goes for #10 as well, but paychecks are on a fixed interval, while text messages come in variable intervals.
#8. The word “average” is crucial: donations can vary tremendously, so the average is no guarantee of a fixed ratio of donations every eighth house.

37. Reinforcement Schedules
RATIO (reinforcement is contingent upon # of responses by organism)
INTERVAL (reinforcement in contingent upon time)
FIXED(reinforcement of a behavior follows a set constant pattern) FR FI
VARIABLE (reinforcement is unpredictable) VR VI

38. Ratio Schedules Fixed Ratio Variable Ratio
Provides a reinforcement after a SET number of responses.
Provides a reinforcement after a RANDOM number of responses.
Very hard to get acquisition but also very resistant to extinction.
Fixed Ratio- She gets a manicure for every 5 pounds she loses.

39. Interval Schedules Fixed Interval Variable Interval
Requires a SET amount of time to elapse before giving the reinforcement.
Requires a RANDOM amount of time to elapse before giving the reinforcement.
Very hard to get acquisition but also very resistant to extinction.
Fixed Interval: She gets a manicure for every 7 days she stays on her diet.

40. Take out your HW Packet page 25 HW: Day 2 – Baseline
Aim: What are some limitations of operant conditioning? What are cognitive views of learning
Do Now:
Take out your HW Packet page 25
HW:
Day 2 – Baseline
Read Packet pps. 39 – 44 and answer Q’s on Handout (Maps in Your Mind)

41. Reinforcement Schedules
Acquisition of a behavior is best under a continuous schedule. Acquisition of a behavior is slower with a partial reinforcement schedule.
Ratio schedules causes a greater response rate.
Variable schedules are most resistant to extinction
Immediate reinforcement is more effective than delayed.

42. Lets imagine I use operant conditioning to encourage my kids to get good grades.
Give an example of the following terms.
Acquisition
Extinction
Spontaneous Recovery
Generalization
Discrimination
*Note – These terms happen in operant AND in classical too!

43. The Effectiveness of Punishment
When/Why it is ineffective?
Creates fear
Creates a feeling of helplessness
Increases aggression
Does not guide or model correct behavior
most misbehavior is hard to punish right away
Administered inappropriately
subject responds with fear, rage or anxiety
effectiveness is temporary
contains little info
can end up being reinforcing

44. Applying operant conditioning to parenting Problems with Physical Punishment
Punished behaviors may simply be suppressed, and restart when the punishment is over.
Instead of learning behaviors, the child may learn to discriminate among situations, and avoid those in which punishment might occur.
Instead of behaviors, the child might learn an attitude of fear or hatred, which can interfere with learning. This can generalize to a fear/hatred of all adults or many settings.
Physical punishment models aggression and control as a method of dealing with problems.
Click to reveal bullets.
Instructor: you can point out to students that these points, especially the first three, are not reasons why physical punishment is morally wrong. Instead, they are reasons why physical punishment doesn’t work well as an operant conditioning technique. The fourth/last point is also an example of ineffectiveness if (presumably) the parent is trying to teach prosocial behavior.

45. When is punishment effective?
Punishment works best in natural settings when we encounter punishing consequences from actions such as reaching into a fire.
In that case, operant conditioning helps us to avoid dangers.
Punishment is less effective when we try to artificially create punishing consequences for other’s choices;
 Severity of punishments is not as helpful as making the punishments immediate and certain.
Click to reveal bullets.
Instructor: an overall answer to the question in the slide title might be that punishment works best when it approximates the way we naturally encounter immediate consequences (less well when the only consequence we encounter is a distant, delayed, possible threat).
You can ask students, testing both their reading and their thinking: “Does the final sentence suggest any changes to our criminal justice system?” The text reviews an example of how drunk driving was reduced not by harsh sentencing, but by better patrolling (more certainty of consequences) and by arresting people right away (more immediate consequences). Another study applied this insight to traffic enforcement. Immediate feedback, even signs showing your speed, work better to reduce traffic violations than punishments that were more severe but delayed and inconsistent.

46. Effective Punishment
Should be swift and brief – don’t wait too long after the undesired behavior to punish
CONSISTENT – punish EVERY time undesired behavior occurs; when it fails to occur the effect can be rewarding
Target the behavior, not the person
Don’t send mixed messages – ex: spanking a child for hitting his/her sibling
Negative punishment (taking away privileges) works better than positive punishment (adding pain)
Explain reasoning and linked with info about appropriate behavior followed by reinforcement of appropriate behavior

47. Learning, Rewards, and Motivation
Intrinsic motivation refers to the desire to perform a behavior well for its own sake. The reward is internalized as a feeling of satisfaction.
Extrinsic motivation refers to doing a behavior to receive rewards from others.
Intrinsic motivation can sometimes be reduced by external rewards overjustification effect) , and can be prevented by using continuous reinforcement.
One principle for maintaining behavior is to use as few rewards as possible, and fade the rewards over time.
Click to reveal bullets.
Answer to the question on the slide: the person might lose intrinsic motivation, and might not reliably do the behavior well.
What might happen if we begin to reward a behavior someone was already doing and enjoying?

48. Criticisms of Behaviorism
Deemphasizes the role of internal thoughts and feelings in behavior
Presents humans as lacking free will
Ignores biological predispositions (Instinctual Drift - tendency for an organism to revert to instinctive behavior)

49. Role of Biology in Conditioning
Can a monkey be trained to peck with its nose? No, but a pigeon can.
Can a pigeon be trained to dive underwater? No, but a dolphin can.
Operant conditioning encounters biological tendencies and limits that are difficult to override.
What can we most easily train a dog to do based on natural tendencies?
detecting scents?
climbing and balancing?
putting on clothes?
Just like classical conditioning, animals more easily learn behaviors that are natural or conducive to survival during operant conditioning.
Click to reveal bullets.
Answer to the question on the slide: animals can be trained to do actions which are less biologically related to survival and mating, but it’s difficult, and behaviors tend to drift toward biologically natural behaviors.

50. Limitations of Operant Conditioning
TASK:
Read about the study on p. 28 of the packet. (Overjustification Effect)
What do you think happened? Why? Which group do you think spent the most time drawing in the next room?

51. Overjustification Effect
Reliance on external incentives can diminish intrinsic motivation.
Intrinsic
reinforcer
Extrinsic
reinforcer

52. Premack Principle
A more preferred activity can be used to reinforce a less preferred activity.
High probability behaviors (those the organism performs under conditions of free choice) could be used to reinforce low probability behaviors.
Therefore what reinforcer could a parent give to a child if they want them to…
Clean their room?
Eat their vegetables?

53. Token Economy
Every time a desired behavior is performed, a token is given.
They can trade tokens in for a variety of prizes (reinforcers)
Used in homes, prisons, mental institutions and schools.

54. More Operant Conditioning Applications
Parenting
Rewarding small improvements toward desired behaviors works better than expecting complete success, and also works better than punishing problem behaviors.
Giving in to temper tantrums stops them in the short run but increases them in the long run.
Self-Improvement
Reward yourself for steps you take toward your goals. As you establish good habits, then make your rewards more infrequent (intermittent).
Click to reveal bullets.
Instructor: there is only one self-improvement strategy rather than the four in the text, because the second one (monitoring) has less to do with operant conditioning, and recent research has shown the first one (announcing your goals to friends) has shown to be incorrect. I combined the third and fourth points.
[Announcing goals apparently creates a false sense of accomplishment that reduces motivation. This is called the substitution effect or the Gollwitzer effect (after psychologist Peter Gollwitzer). I let the author know about this.]

55. Fun Theory

56. Behavior Modification
“A New Toy for Teddy” US UR CS CR
Operant Conditioning
Negative Reinforcement
Escape/ Avoidance Conditioning
***Was it ethical? Would you have recommended this treatment?

57. Applications of Operant Conditioning
School: long before tablet computers, B.F. Skinner proposed machines that would reinforce students for correct responses, allowing students to improve at different rates and work on different learning goals.
Sports: athletes improve most in the shaping approach in which they are reinforced for performance that comes closer and closer to the target skill (e.g., hitting pitches that are progressively faster).
Work: some companies make pay a function of performance or company profit rather than seniority; they target more specific behaviors to reinforce.
Click to reveal three applications.
Instructor: you might mention another application--kids with severe, nonverbal autism at an early age are sometimes treated with Applied Behavioral Analysis with Discrete Trial Training. In this approach, specific behaviors such as sitting to eat, making eye contact, or using words to get needs met, are targets for daily sessions of repeated shaping and reinforcement.

58. Boy Who Wouldn’t Eat
How would the 2 Factor Theory explain the development of this disorder?
How would you use classical or operant conditioning to modify this behavior?
BEFORE
AFTER