PSY 445: Learning & Memory Chapter 5: Instrumental Conditioning: Nonreward, Punishment, & Avoidance

Aversive-Outcome Learning Nonreward Extinction Omission Punishment Avoidance learning Negative Reinforcement

Extinction The non-reinforcement of a previously reinforced response The result is the decrease in the strength of the initially reinforced response

Extinction Hi Strength of R Lo Acquisition R  No S*

Omission A selected response prevents a positive reinforcer from occurring Consists of the removal of a stimulus (usually one that is considered to be pleasant) following a response that leads to a decrease in the future strength of that response R ------------------------------- S (tease sister) (no computer games)

Extinction Side Effects… Unpleasant Emotional Effects Frustration; Aggression Extinction Burst Temporary increase in the nonreinforced behavior Alessandri, Sullivan, & Lewis (1990) Spontaneous Recovery After a delay the response recovers Infant mobile  See next slide

Extinction Side Effects… Unpleasant Emotional Effects Frustration; Aggression Extinction Burst Temporary increase in the nonreinforced behavior Alessandri, Sullivan, & Lewis (1990) Spontaneous Recovery After a delay the response recovers Infant mobile  See next slide

Spontaneous Recovery Hi Time passes Strength of R Lo Acquistion Extinction S+ occurs

A more positive side effect… When old behavior doesn’t work, new behaviors may replace it Neuringer, Kornell, & Olafs (2001) Procedure Rats trained to press three specific levers (of five present) to get food pellet reward Extinction phase food pellets withheld Results Rats tried new sequence of lever pushing to see if they could get reward Interpretation Real-world adaptation: If food is not obtained with one behavior, survival depends on us to be flexible enough to try novel and creative behavior This runs counter to Thorndike’s old interpretation where cats continued with stereotyped behavior

The Partial Reinforcement Extinction Effect (PREE) Extinction is slower if a partial reinforcement schedule (PRS), rather than a continuous reinforcement schedule (CRS) was in effect before extinction Acquisition: CRS rats ran faster to look for reward Extinction: PRS rats ran faster to look for reward

The Partial Reinforcement Extinction Effect (PREE) Explanations for resistance Discrimination Hypothesis For CRS rats, the onset of extinction is readily discriminated from the previous continuous schedule of rewards; something has changed For PRS rats, the first extinction trial is not considered permanent Frustration Hypothesis For PRS rats, the frustrating aftereffects of nonreward become associated with the subsequent occurrence of reward Frustration experiences on one trial becomes SD for reward

The Partial Reinforcement Extinction Effect (PREE) Sequential Hypothesis At the start of a new trial, PRS rats remember the outcome of the previous trial and associates it with the outcome of the current trial Less trials needed here than with frustration idea (memory quicker to form than is frustration) Trials can be spaced further apart as memory persists longer than frustration

Punishment An aversive stimulus is presented immediately after a behavior in order to decrease the future probability of that behavior (or strength of this response) For example: when your dog chases a car R --------------------------------------- S (run s into traffic) (scolded)

When does punishment work? Punishing stimuli Shock used in most animal studies Response-contingent punishment Be sure specific response is being punished Intensity Strong better than weak; don’t use gradual increases Delay of punishment More effective when applied immediately Solomon, Turner, & Lessac (1968) Dogs punished immediately, after 5 seconds,; after 15 seconds Immediate worked best; fifteen second delay was least effective

When does punishment work? Schedule of punishment Continuous schedule is generally better than partial schedule; however, spontaneous recovery is more likely Incompatible responses Be careful to punish in a way that will lead to the desired outcome Unfortunately, at times the punishing stimulus can elicit behaviors that are incompatible with the desired outcome Concurrent reinforcement The effects of punishment can be neutralized if positive reinforcement of the inappropriate behavior occurs along with punishment

When does punishment work? Providing a verbal rationale Less severe punishment is enhanced by including an explanation Cheyne, Goyeche, & Walters (1969) Procedure Different types of punishment with or without verbal rationale were given to children attempting to reach for a toy Results No rationale most severe punishment is most effect With rationale lesser punishment worked significantly better See next slide 

When does punishment work? Interpretation Strong emotional responses that accompany punishment may interfere with the ability to attend to and/or learn from the situation (narrowing of attention) Results  Cheyne, Goyeche, & Walters (1969)

When does punishment work? Individual differences Some people or animals may be less reactive to punishment than others

Unwanted side effects of punishment Conditioned fear avoidance The place where punishment takes place or the person administering the punishment may become CS leading to CR Aggression Punishment-aggression hypothesis Miller (1948): rats exhibited shock-aggression behavior Aggression is reinforced Generalized aggression Paradoxical rewarding effects The pairing of a punishing stimulus with a positive reinforcer can convert the punisher into a secondary reinforcer The punishing stimulus can inadvertently enforce behavior rather than suppress in it Masserman (1943)

Paradoxical rewarding effects Masserman (1943) Procedure Trained cats to lever press for unpleasant blasts of air Phase 1: Lever press for reward (food) Phase 2: Blasts only occasionally and mild Phase 3: Gradually increased air blasts and decreased food rewards Results Cats would bar press for punishment Interpretation Knowing the history of an organism can explain irrational behavior

Punishment or Nonreward? Frustration produced by nonreward can elicit aggression (similar to punishment)

Punishment or Nonreward? Does punishment work in everyday life? Are punished children better behaved? Methodological issues Different types of punishment Who decides if the child is misbehaving Conflicting research Some say more problems; others say neither benefits nor deficits Correlational issues Not necessarily cause and effect relationship

Persistence Resistance to extinction is the extent to which responding continues after an extinction procedure has begun Persistent responding = high resistance to extinction Responding stops quickly = low resistance to extinction

Persistence Schedule of reinforcement Partial during acquisition Magnitude of the reinforcer Smaller during acquisition Delay of the reinforcer Delayed during acquisition

Persistence Generalized Persistence In certain cases, responding is resilient Can carry over from one situation to another Eisenberger et al. (1979) Procedure Depressed patients asked to perform chores Phase 1: Gratitude was either on continuous or partial reinforcement schedule Phase 2: New person requests favor and shows gratitude on partial reinforcement schedule Results Patients spent more time on a chore (punching computer cards) if initially on partial reinforcement schedule Interpretation Persistence generalized beyond initial task to another task and from another person

Avoidance Learning An instrumental response escapes or prevents an aversive stimulus Often referred to as negative reinforcement because the instrumental response increases in frequency Two types of Negative Reinforcement Escape Conditioning When a behavior has terminated an aversive stimulus Avoidance Conditioning When a behavior has prevented an aversive stimulus

Avoidance Learning Typical procedure Learning task begins with warning signal which is followed by shock On succeeding trials, selected instrumental response prevents aversive stimulus and terminates both the warning signal and trial Shuttle Box 

Avoidance Learning Active Avoidance Active performance of a certain response prevent the aversive stimulus Passive Avoidance Withholding a response prevents the aversive outcome

Theories of Avoidance Learning Conditioning Theory Watson-Mowrer Two-Process Theory Cognitive Theory Functional Approach

Conditioning Theory Watson-Mowrer Two-Process Theory (Mowrer , 1947) Avoidance learning involved two processes - (1) classical conditioning and (2) instrumental conditioning Process 1: Dangerous, painful, aversive stimuli (US) causes an innate fear response (UR) Other stimuli present at the time get associated with fear through classical conditioning. When these other stimuli (CSs) are encountered again, they evoke a fear response (CR) Process 2: The presence of fear and all of its visceral effects is aversive Any response that removes these fear-evoking stimuli will be negatively reinforced

Conditioning Theory Solomon & Wynne (1953) Illustrates Two-Process Theory Shuttle box consists of: 2 chambers separated by a barrier several inches high Could freely move between chambers Separate lights Metal floor that could be electrified to deliver shock

Conditioning Theory Solomon & Wynne (1953) Procedure Experimental Session – 10 trials each Dog placed in one compartment of a shuttle box After a time, light in the compartment that dog was in went off, while other light stayed on 10 seconds later, the dog received a shock through the floor until jumped over barrier Measured response latency Results Escape first: First few responses were usually longer than 10 seconds. So dog was getting shocked and then escaping. Avoidance response: By roughly the 5th trial, dogs response latency was less than 10 seconds. So dog never received shock in these cases. Interpretation Both processes are involved

Conditioning Theory Limitations Apparently endless persistence in avoidance behavior Fear to warnings never seem to go away Examples: people with phobias Seems to fit better with classical conditioning ideas

Cognitive Theory Eliminating avoidance behavior requires modifying two expectations: Response - expect to avoid aversive outcome No response – expect to get shocked Subject must learn that the warning stimulus no longer signals danger

Functional Approach Species-Specific Defense Responses (SSDRs) Studying the natural behaviors of the organism’s reactions to threats in the wild Innate responses that are primed in a fearful or threatening situation These behaviors are witnessed in lab setting (for example, mice in fearful situations will “freeze”)

Approach-Avoidance Conflict When a behavior has two opposing outcomes reward and punishment what is the result? Relative intensity Proximity to the consequence (physically or temporally)

Approach or Avoidance as a Coping Response Approach Coping Confronting the stressor; problem-focused coping Avoidance Coping Shutting down and trying to deny or suppress the unpleasant thoughts and feelings; repressive coping Distraction can be an adaptive form of avoidance coping

Approach or Avoidance as a Coping Response Averill & Rosenn (1972) Shock-avoidance experiment Procedure Audiotape with two tracks Track 1: Music Track 2: Tone of the impending shock Results Some students listened to music exclusively Interpretation Sometimes the anticipation of unpleasantness is worse than the actual aversive stimulus

Approach or Avoidance as a Coping Response Which style is best? It depends… Is stressor controllable or not? Habituation and counterconditioning process processes are involved Are thoughts intrusive? Ironic processes may be involved (Wegner, 1987)

Learned Helplessness Learning that there is an explicit lack of contingency between responses and an aversive outcome

Learned Helplessness Seligman & Maier (1967) Procedure Phase 1: Dogs receive either: escapable shock (press button with nose), inescapable shock, no shock Phase 2: All placed in shuttle box (two components) Light warned of shock on one side Escape shock by moving to other side Results Dogs that had been exposed to escapable shock and no shock quickly learned to escape Inescapable shock dogs “quietly whined" Interpretation Inescapable shock dogs developed a perceived loss of control

Learned Helplessness Hiroto (1974) Procedure College students exposed to loud tone pulses that were “unpleasant but not harmful” Phase 1: Escapable group and inescapable group Phase 2: Both groups now could escape tone Results Inescapable group was slow to respond in Phase 2 Interpretation Inescapable college students developed a perceived loss of control

Extensions of Learned Helplessness Depression Symptoms of both helpless and depressed mimic each other – reductions in activity; loss of motivation; disruptions in eating, sleeping, and sexual behavior Assertiveness training can be beneficial to these individuals Causal Attribution Theory This is the idea that helplessness is ultimately determined by the belief that our behavior is ineffectual in determining what happens to us Insofar as their causal attributions are concerned, helpless individuals have an external locus of control

Extensions of Learned Helplessness Classroom Education Locus of control is involved here as well Physical Health Correlational findings associate negative attitudes with physical health problems

Neuroscience and Aversive Learning Amygdala is involved in classical conditioning of fear Strong evidence from studies on both animals and humans

Applications of Aversive-Learning Contingencies Pet Containment Systems Invisible Fencing technique applies Watson- Mowrer theory of conditioning Potential to provoke aggression is a concern Treatment of Obsessive-Compulsive Disorder Stimulus extinction and response prevention have been used to treat these individuals Usually combined with other methods (cognitive therapies; pharmacological treatments)

Credits: Some slides prepared with the help of the following websites: www.radford.edu/~pjackson/ExtinctIC.ppt www.public.coe.edu/~mbaker/baker/learning/lectures http://pjackson.asp.radford.edu/punishment3.pdf http://web.mnstate.edu/malonech/Psy342/Terry%20Notes/Instru mental%20Cond.%20T4T5.htm