1. PSY 445: Learning & Memory Chapter 5: Instrumental Conditioning:
Nonreward, Punishment, & Avoidance

2. Aversive-Outcome Learning
Nonreward
Extinction
Omission
Punishment
Avoidance learning
Negative Reinforcement

3. Extinction The non-reinforcement of a previously reinforced response
The result is the decrease in the strength of the initially reinforced response

4. Extinction Hi
Strength
of R Lo
Acquisition R  No S*

5. Omission
A selected response prevents a positive reinforcer from occurring
Consists of the removal of a stimulus (usually one that is considered to be pleasant) following a response that leads to a decrease in the future strength of that response
R  S
(tease sister) (no computer games)

6. Extinction Side Effects…
Unpleasant Emotional Effects
Frustration; Aggression
Extinction Burst
Temporary increase in the nonreinforced behavior
Alessandri, Sullivan, & Lewis (1990)
Spontaneous Recovery
After a delay the response recovers
Infant mobile 
See next slide

7. Extinction Side Effects…
Unpleasant Emotional Effects
Frustration; Aggression
Extinction Burst
Temporary increase in the nonreinforced behavior
Alessandri, Sullivan, & Lewis (1990)
Spontaneous Recovery
After a delay the response recovers
Infant mobile 
See next slide

8. Spontaneous Recovery Hi Time passes Strength of R Lo
Acquistion Extinction S+ occurs

9. A more positive side effect…
When old behavior doesn’t work, new behaviors may replace it
Neuringer, Kornell, & Olafs (2001)
Procedure
Rats trained to press three specific levers (of five present) to get food pellet reward
Extinction phase food pellets withheld
Results
Rats tried new sequence of lever pushing to see if they could get reward
Interpretation
Real-world adaptation: If food is not obtained with one behavior, survival depends on us to be flexible enough to try novel and creative behavior
This runs counter to Thorndike’s old interpretation where cats continued with stereotyped behavior

10. The Partial Reinforcement Extinction Effect (PREE)
Extinction is slower if a partial reinforcement schedule (PRS), rather than a continuous reinforcement schedule (CRS) was in effect before extinction
Acquisition: CRS rats ran faster to look for reward
Extinction: PRS rats ran faster to look for reward

11. The Partial Reinforcement Extinction Effect (PREE)
Explanations for resistance
Discrimination Hypothesis
For CRS rats, the onset of extinction is readily discriminated from the previous continuous schedule of rewards; something has changed
For PRS rats, the first extinction trial is not considered permanent
Frustration Hypothesis
For PRS rats, the frustrating aftereffects of nonreward become associated with the subsequent occurrence of reward
Frustration experiences on one trial becomes SD for reward

12. The Partial Reinforcement Extinction Effect (PREE)
Sequential Hypothesis
At the start of a new trial, PRS rats remember the outcome of the previous trial and associates it with the outcome of the current trial
Less trials needed here than with frustration idea (memory quicker to form than is frustration)
Trials can be spaced further apart as memory persists longer than frustration

13. Punishment
An aversive stimulus is presented immediately after a behavior in order to decrease the future probability of that behavior (or strength of this response)
For example: when your dog chases a car
R  S
(run s into traffic) (scolded)

14. When does punishment work?
Punishing stimuli
Shock used in most animal studies
Response-contingent punishment
Be sure specific response is being punished
Intensity
Strong better than weak; don’t use gradual increases
Delay of punishment
More effective when applied immediately
Solomon, Turner, & Lessac (1968)
Dogs punished immediately, after 5 seconds,; after 15 seconds
Immediate worked best; fifteen second delay was least effective

15. When does punishment work?
Schedule of punishment
Continuous schedule is generally better than partial schedule; however, spontaneous recovery is more likely
Incompatible responses
Be careful to punish in a way that will lead to the desired outcome
Unfortunately, at times the punishing stimulus can elicit behaviors that are incompatible with the desired outcome
Concurrent reinforcement
The effects of punishment can be neutralized if positive reinforcement of the inappropriate behavior occurs along with punishment

16. When does punishment work?
Providing a verbal rationale
Less severe punishment is enhanced by including an explanation
Cheyne, Goyeche, & Walters (1969)
Procedure
Different types of punishment with or without verbal rationale were given to children attempting to reach for a toy
Results
No rationale most severe punishment is most effect
With rationale lesser punishment worked significantly better
See next slide 

17. When does punishment work?
Interpretation
Strong emotional responses that accompany punishment may interfere with the ability to attend to and/or learn from the situation (narrowing of attention)
Results 
Cheyne, Goyeche, & Walters (1969)

18. When does punishment work?
Individual differences
Some people or animals may be less reactive to punishment than others

19. Unwanted side effects of punishment
Conditioned fear avoidance
The place where punishment takes place or the person administering the punishment may become CS leading to CR
Aggression
Punishment-aggression hypothesis
Miller (1948): rats exhibited shock-aggression behavior
Aggression is reinforced
Generalized aggression
Paradoxical rewarding effects
The pairing of a punishing stimulus with a positive reinforcer can convert the punisher into a secondary reinforcer
The punishing stimulus can inadvertently enforce behavior rather than suppress in it
Masserman (1943)

20. Paradoxical rewarding effects
Masserman (1943)
Procedure
Trained cats to lever press for unpleasant blasts of air
Phase 1: Lever press for reward (food)
Phase 2: Blasts only occasionally and mild
Phase 3: Gradually increased air blasts and decreased food rewards
Results
Cats would bar press for punishment
Interpretation
Knowing the history of an organism can explain irrational behavior

21. Punishment or Nonreward?
Frustration produced by nonreward can elicit aggression (similar to punishment)

22. Punishment or Nonreward?
Does punishment work in everyday life?
Are punished children better behaved?
Methodological issues
Different types of punishment
Who decides if the child is misbehaving
Conflicting research
Some say more problems; others say neither benefits nor deficits
Correlational issues
Not necessarily cause and effect relationship

23. Persistence
Resistance to extinction is the extent to which responding continues after an extinction procedure has begun
Persistent responding = high resistance to extinction
Responding stops quickly = low resistance to extinction

24. Persistence Schedule of reinforcement Partial during acquisition
Magnitude of the reinforcer
Smaller during acquisition
Delay of the reinforcer
Delayed during acquisition

25. Persistence Generalized Persistence
In certain cases, responding is resilient
Can carry over from one situation to another
Eisenberger et al. (1979)
Procedure
Depressed patients asked to perform chores
Phase 1: Gratitude was either on continuous or partial reinforcement schedule
Phase 2: New person requests favor and shows gratitude on partial reinforcement schedule
Results
Patients spent more time on a chore (punching computer cards) if initially on partial reinforcement schedule
Interpretation
Persistence generalized beyond initial task to another task and from another person

26. Avoidance Learning
An instrumental response escapes or prevents an aversive stimulus
Often referred to as negative reinforcement because the instrumental response increases in frequency
Two types of Negative Reinforcement
Escape Conditioning
When a behavior has terminated an aversive stimulus
Avoidance Conditioning
When a behavior has prevented an aversive stimulus

27. Avoidance Learning Typical procedure
Learning task begins with warning signal which is followed by shock
On succeeding trials, selected instrumental response prevents aversive stimulus and terminates both the warning signal and trial
Shuttle Box 

28. Avoidance Learning Active Avoidance
Active performance of a certain response prevent the aversive stimulus
Passive Avoidance
Withholding a response prevents the aversive outcome

29. Theories of Avoidance Learning
Conditioning Theory
Watson-Mowrer Two-Process Theory
Cognitive Theory
Functional Approach

30. Conditioning Theory Watson-Mowrer Two-Process Theory (Mowrer , 1947)
Avoidance learning involved two processes - (1) classical conditioning and (2) instrumental conditioning
Process 1:
Dangerous, painful, aversive stimuli (US) causes an innate fear response (UR)
Other stimuli present at the time get associated with fear through classical conditioning.
When these other stimuli (CSs) are encountered again, they evoke a fear response (CR)
Process 2:
The presence of fear and all of its visceral effects is aversive
Any response that removes these fear-evoking stimuli will be negatively reinforced

31. Conditioning Theory Solomon & Wynne (1953)
Illustrates Two-Process Theory
Shuttle box consists of:
2 chambers separated by a barrier several inches high
Could freely move between chambers
Separate lights
Metal floor that could be electrified to deliver shock

32. Conditioning Theory Solomon & Wynne (1953) Procedure
Experimental Session – 10 trials each
Dog placed in one compartment of a shuttle box
After a time, light in the compartment that dog was in went off, while other light stayed on
10 seconds later, the dog received a shock through the floor until jumped over barrier
Measured response latency
Results
Escape first: First few responses were usually longer than 10 seconds. So dog was getting shocked and then escaping.
Avoidance response: By roughly the 5th trial, dogs response latency was less than 10 seconds. So dog never received shock in these cases.
Interpretation
Both processes are involved

33. Conditioning Theory Limitations
Apparently endless persistence in avoidance behavior
Fear to warnings never seem to go away
Examples: people with phobias
Seems to fit better with classical conditioning ideas

34. Cognitive Theory
Eliminating avoidance behavior requires modifying two expectations:
Response - expect to avoid aversive outcome
No response – expect to get shocked
Subject must learn that the warning stimulus no longer signals danger

35. Functional Approach Species-Specific Defense Responses (SSDRs)
Studying the natural behaviors of the organism’s reactions to threats in the wild
Innate responses that are primed in a fearful or threatening situation
These behaviors are witnessed in lab setting (for example, mice in fearful situations will “freeze”)

36. Approach-Avoidance Conflict
When a behavior has two opposing outcomes reward and punishment what is the result?
Relative intensity
Proximity to the consequence (physically or temporally)

37. Approach or Avoidance as a Coping Response
Approach Coping
Confronting the stressor; problem-focused coping
Avoidance Coping
Shutting down and trying to deny or suppress the unpleasant thoughts and feelings; repressive coping
Distraction can be an adaptive form of avoidance coping

38. Approach or Avoidance as a Coping Response
Averill & Rosenn (1972)
Shock-avoidance experiment
Procedure
Audiotape with two tracks
Track 1: Music
Track 2: Tone of the impending shock
Results
Some students listened to music exclusively
Interpretation
Sometimes the anticipation of unpleasantness is worse than the actual aversive stimulus

39. Approach or Avoidance as a Coping Response
Which style is best? It depends…
Is stressor controllable or not?
Habituation and counterconditioning process processes are involved
Are thoughts intrusive?
Ironic processes may be involved (Wegner, 1987)

40. Learned Helplessness
Learning that there is an explicit lack of contingency between responses and an aversive outcome

41. Learned Helplessness Seligman & Maier (1967) Procedure
Phase 1: Dogs receive either: escapable shock (press button with nose), inescapable shock, no shock
Phase 2: All placed in shuttle box (two components)
Light warned of shock on one side
Escape shock by moving to other side
Results
Dogs that had been exposed to escapable shock and no shock quickly learned to escape
Inescapable shock dogs “quietly whined"
Interpretation
Inescapable shock dogs developed a perceived loss of control

42. Learned Helplessness Hiroto (1974) Procedure
College students exposed to loud tone pulses that were “unpleasant but not harmful”
Phase 1: Escapable group and inescapable group
Phase 2: Both groups now could escape tone
Results
Inescapable group was slow to respond in Phase 2
Interpretation
Inescapable college students developed a perceived loss of control

43. Extensions of Learned Helplessness
Depression
Symptoms of both helpless and depressed mimic each other – reductions in activity; loss of motivation; disruptions in eating, sleeping, and sexual behavior
Assertiveness training can be beneficial to these individuals
Causal Attribution Theory
This is the idea that helplessness is ultimately determined by the belief that our behavior is ineffectual in determining what happens to us
Insofar as their causal attributions are concerned, helpless individuals have an external locus of control

44. Extensions of Learned Helplessness
Classroom Education
Locus of control is involved here as well
Physical Health
Correlational findings associate negative attitudes with physical health problems

45. Neuroscience and Aversive Learning
Amygdala is involved in classical conditioning of fear
Strong evidence from studies on both animals and humans

46. Applications of Aversive-Learning Contingencies
Pet Containment Systems
Invisible Fencing technique applies Watson- Mowrer theory of conditioning
Potential to provoke aggression is a concern
Treatment of Obsessive-Compulsive Disorder
Stimulus extinction and response prevention have been used to treat these individuals
Usually combined with other methods (cognitive therapies; pharmacological treatments)

47. Credits: Some slides prepared with the help of the following websites:
mental%20Cond.%20T4T5.htm