1. Chapter 10 – Aversive Control: Avoidance and Punishment
Outline
Active Avoidance vs. Passive Avoidance
Negative Reinforcement vs. Positive Punishment
Avoidance
Origins of the Study of Avoidance
Discriminated Avoidance (Signaled Avoidance)
Mowrer’s Two-Factor Theory of Avoidance
Experimental Analysis of Avoidance Behavior
Acquired Drive Experiements
Extinction of Avoidance
Nondiscriminated (Free-Operant) Avoidance
Punishment
What is Punishing?
Is Punishment Effective?
Rules of Punishment

2. Avoidance procedures increase the occurrence of instrumental behavior
Active avoidance
Negative Reinforcement (Escape or Avoidance)
Learn to make a response to avoid a negative outcome
Punishment procedures suppress instrumental responding
Passive avoidance
Positive Punishment
Learn to withhold responding to avoid a negative outcome.

3. Origins of the study of avoidance
Vladimir Becherev (1913)
Participants were asked to put their finger on a metal plate.
tone (CS) was followed by a shock (US) through the metal plate
Participants quickly learned to pick up their finger (CR) when they heard the tone (CS).
This was considered to be a standard example of classical conditioning.
But it was different
Removing finger cancelled the US
avoidance

4. Brogden, Lipman, and Culler (1938)
Directly compared classical conditioning to avoidance learning
Guinea pigs in a running wheel.
CS- tone
US – shock
The shock stimulated the guinea pigs to run (UR)
Gp1 Gp2
Classical Avoidance
US always followed US followed the tone
the tone unless the Guinea’s ran
The Avoidance group ran much more than the classically conditioned animals.

5. Figure 10.1 – Modern running wheel for rodents.

7. Discriminated Avoidance procedure (signaled avoidance procedure)
There is a warning stimulus (CS)
What happens after the CS depends upon what the animal does.
Escape Trial
Don’t respond fast enough
US is presented until response is made
Successful Avoidance Trial
Respond quickly enough
CS is turned off and there is no US on that trial.

8. Figure 10.3 – Diagram of the discriminated, or signaled, avoidance procedure.

9. During early training most trials are escape trials
eventually the animals learn to make the avoidance response, and then avoidance behaviors take over.
These experiments have often been performed in shuttle boxes.

11. Shuttle avoidance Two-way One-way move back and forth.
always start each trial on one side
move to the other.

12. Mowrer’s two factor theory of Avoidance.
The problem for avoidance tasks is that once animals are performing well, the outcome (shock) doesn’t occur anymore
Looks like extinction
So what motivates responding?
Pavlovian (emotional conditioning)
Signaled
CS – US
Instrumental 
After fear is conditioned the second thing that is learned is the instrumental response (jump barrier).
What motivates responding?
Negative RF
escape from conditioned fear.  

13. Experimental Analysis of Avoidance Behavior Acquired drive experiments
In the typical avoidance procedure the classical conditioning, and instrumental conditioning, occur simultaneously.
But if two-factor theory is correct it should be possible to train these two kinds of learning separately.
1) condition fear to a CS with a pure classical conditioning procedure.
CS (tone) --> US (shock)  
CR (fear)
2) the animals are periodically presented with the CS, but an instrumental response can prevent it.
No shocks are delivered in phase 2.

14. Classical conditioning
Brown and Jacobs (1949).
Classical conditioning
Confined rats in one side of a shuttle box – the shuttle opening was blocked. 
Light/tone (CS) --> shock (US).
22 trials.
Instrumental conditioning
The shuttle box opening was opened. 
CS came on, and remained on until the rats went to the other side – escape.
The animal was removed until the next trial.
Over time the shock conditioned animals learned to cross over to the other side sooner after the initiation of the CS.
Do we need a control group?

15. Extinction of Avoidance?
Solomon, Kamin, and Wynne (1953)
One dog avoided shock for 650 straight trials after escaping a few.
Is it possible to extinguish?
Flooding or Response Prevention
Participant is prevented from making the instrumental response during CS
The US is omitted (extinction)
Block off door to shuttle box
CS alone
Has clinical implications
Help people extinguish pathological avoidance
Learn to avoid anxiety producing situations.

16. Nondiscriminated (Free-Operant) Avoidance
Executive Monkey
Press key every 15 seconds to avoid shock.
S-S interval – time between shocks if you don’t respond
R-S interval – period of safety created if you do respond
Don’t have to be the same.
S-S 8s
R-S 15s.
you will receive a shock every 8s unless you respond.
If you do respond you buy yourself 15s in the absence of shock.

17. Chimps have been trained with several schedules to keep track of to avoid shock (Koestler & Barker, 1965).
respond every 5-s on a button to avoid shock (Free operant avoidance)
any key that lit up with a light (Sd) had to be pressed within one second or they would be shocked (Signaled Avoidance).
a separate unlit key had depressed within one second of a second Sd ( a tone; Signaled Avoidance)
The Chimps were able to keep track of all of this at once
motivated by fear?

18. So Fear motivates responding?
Kamin, Brimer, and Black ran an experiment that showed that this may not be the case.
Conditioned Suppression
Phase 1
Rats trained to barpress for food
Phase 2
Then put into a shuttle box where a tone Sd set the occasion to jump the barrier to avoid shock
There were four groups (they differ in amount of training)
group 1 = avoid shock by jumping barrier on 1 trial
group2 = avoid shock by jumping barrier on 3 consecutive trials
group 3 = avoid shock by jumping barrier on 9 consecutive trials
group 4 = avoid shock by jumping barrier on 27 consecutive trials
Test
returned to the operant chamber
continued to barpress for food.
The tone was sounded

19. Think of an air traffic controller.
Rats that had reached the 1, 3, or 9 consecutive trial criterion, showed considerable suppression when the tone sounded.
Fear tone.
surprisingly the rats that had had 27 consecutive trials of avoidance did not suppress responding.
It seems that increasing experience with the situation lessens the fear.
especially because the animals have learned they have control over the situation.
Think of an air traffic controller.
are they motivated by negative reinforcement?
are they afraid all of the time?
are they only afraid when they think they have made a mistake? 
Nevertheless Two- Factor theory is the theory of avoidance against which all others are compared.

20. As a society we tend to be very interested in punishment
As we have already discussed. Punishment is some manipulation that causes a decrease in responding 
As a society we tend to be very interested in punishment
Crime
Children
Is Punishment effective?
Skinner initially argued that Punishment was not very effective
Possibly came to this conclusion because he was using weak punishers
Electronic slap of the paw
It has been shown that punishment can be effective if the correct conditions are met
Domjan’s example
Ticket for speeding
Child sticks fork in electric socket.

21. Experimental Analysis of Punishment
Punishment involves decreasing a behavior
in order to measure it you must initially have a baseline of that behavior
In real life or therapeutic situations punishment is used to decrease some maladaptive behavior
Self injurious behavior
Drug taking
In the lab, we typically first train the animals to perform a behavior
Then introduce punishment to see if it suppresses responding
This can make things complicated
Is responding more influenced by reward or punishment?
It depends
Skinner’s paw slap

22. Characteristics of the Aversive Stimulus and its Method of Introduction
Types of Stimuli
Remember one form of punishment (positive punishment) involves presentation of an aversive stimulus
Shock
Loud noise
slap
Squirt of lemon juice in the mouth

23. negative punishment involves taking away something the organism wants
Loss of positive RF
Time out
Removal of personal freedoms
Can’t get typical RF

24. When is Punishment most effective?
Let’s go through what could be considered the “Rules for Effective Punishment”
This list is backed up by empirical findings from the animal literature
Think about how we often don’t follow these rules in real life though
1)    punishment must be delivered soon after a response to be most effective
“Catch them in the act”
Delayed punishment is far less effective
Can work well with your dog or cat.
Kids?
Criminals?

25. 2) Punishment must be strong as possible to be effective
Mild shock causes only moderate suppression of behavior
The effect of mild shock dissipates with repeated application
Habituation
Intense, longer lasting shocks are far more effective
Kids?
Criminals?

26. 3) Punishment must be delivered consistently to be effective.
Punishment works best if it is applied every time
How does this compare to the effects of partial reinforcement?
Kids?
Criminals?

27. 4) Punishment should be as strong as possible initially
It is not very effective to start off weak and then increase with repeated infractions.
Kids?
Criminals?

28. 5) Response-Contingent versus Response-Independent Aversive Stimulation
Random punishment can suppress responding.
Especially if intense
Learned helplessness
Typically, however, random punishment is not as effective as contingent punishment
With random punishment in a rat you might see some initial decline in responding
but as the animal becomes used to it responding tends to increase

29. 6)    noncontingent punishment decreases the effectiveness of contingent punishment
If animals are shocked at random initially they take longer to associate shock with a specific stimulus, or response.
Phase 1: Random shocks
Phase 2: Lever press = shock
US preexposure effect
Kids?
Criminals?

30. 7)    Punishment is more effective if animal if offered an alternative response to the punished response
Herman and Azrin (1964)
Human smokers
Two levers
Phase1 (Training)
Lever 1: cigarette VI schedule
Lever 2: cigarette VI schedule
Phase 2 (Test)
Condition 1
Lever 1: loud noise; cigarette VI schedule
Lever 2: cigarette VI Schedule
Condition 2
Findings?
How much pressing of lever 1?
Condition 1?
Condition 2?

31. The findings from Herman and Azrin (1964) may seem obvious, but the implications are important
If you want to discourage a particular behavior, you can punish it.
But it would be more effective to punish that behavior while providing the organism alternative reinforcing/reinforced behaviors
Punish playing too much x-box, but supply lots of new books/magazines?
Punish self-injurious behavior, but supply other activities (painting, music, social interaction).
Providing alternative behavioral opportunities can make even mild punishment much more effective.

32. 8) Punishment can come under stimulus control
Discriminative punishment
Responding is punished in the presence of an S+
Light indicates lever pressing will result in shock
Otherwise responding is RF
Animals quickly learn not to respond to the lever when a light is on.
Kids?
Parent present or not
Classroom effects
Teacher present
Substitute/weak teacher
Criminals?
Police car  speeding