Presentation is loading. Please wait.

Presentation is loading. Please wait.

The partial reinforcement extinction effect (PREE) Frode Svartdal University of Tromsø Oct. 2013.

Similar presentations

Presentation on theme: "The partial reinforcement extinction effect (PREE) Frode Svartdal University of Tromsø Oct. 2013."— Presentation transcript:

1 The partial reinforcement extinction effect (PREE) Frode Svartdal University of Tromsø Oct. 2013

2 Extinction: Basics Extinction is defined in terms of a reinforcement process Extinction contingencies The stimulus (S R or US) is discontinued The learning contingency is discontinued Extinction process The conditioned response is reduced (strength, frequency, etc.) Relearning, … not forgetting

3 Catania, 1984) Extinction: Basics Operant conditioning

4 Extinction: Basics Classical conditioning

5 Factors affecting the extinction rate In general: Fast acquisition / high rate of responding  fast extinction Amount of reward High  fast extinction Variability Stimulus Response Reinforcement Some forms of learning do not extinguish (easily) Evaluative conditioning (e.g., Diaz, Ruiz, & Beyens, 2005) = high ext. persistence

6 Factors affecting the extinction rate Partial Reinforcement Extinction Effect Partial (Intermittent) Reinforcement (PRF)  increased extinction response Continuous Reinforcement (CRF)  reduced extinction persistence

7 First demonstrations Operant conditioning; free operant; rats; Skinner (1938) Classical conditioning; blink response; students; Humphreys (1939) 100% 50%

8 Ferster & Culbertson, 1975 Free operant

9 PRF CRF Free operant Compared to CRF: PRF  higher asymptotes more persistent respondingunder extinction EXTINCTION

10 Rats, maze running speed under extinction (Weinstock, 1954) CRF PRF (30%)

11 Classical conditioning (rats): PREE 25% 50% 100% Extinction PRF response rate LOWER than CRF response rate 15%

12 Classical conditioning; eyelid; human subjects (Svartdal & Flaten, in prep.)

13 Operant conditioning; humans; Svartdal, 2003, Exp. 4

14 Conclusions (… preliminary) PREE is a very robust outcome Measures & species Bar pressing, rats Maze running, rats Pecking, pigeons Blink reflex, humans, rabbits … Contingency Operant/instrumental Discrete trial Free operant Classical

15 But… How general is the PREE? Reversed PREE observed under some conditions Generalized PREE observed under some conditions Alternative methods of analysis Nevin (1988): ”PREE is an artefact because of wrong method of analzing extinction performance” Response unit issue PREE or not dependig on how the response is defined (Mowrer & Jones, 1945!

16 Reversed PREE What happens if the subject is exposed to a mixture of PRF and CRF contingencies?

17 Reversed PREE Pavlik & Carlton, 1965: Rats; bar pressing, free operant Gr. 1: Single contingency; CRF Gr. 2: Single contingency; PRF Gr. 3: Two signalled schedules alternated for the same subjects; CRF + PRF

18 Reversed PREE Conventional PREE

19 Reversed PREE Reversed PREE

20 Reversed PREE Pavlik & Carlton (1965): Single reinforcement schedules (CRF vs. PRF) in between- groups experiments  PREE Two schedules (CRF vs. PRF) for the same subjects  Reversed PREE Other research Reversed PREE observed Generalized PREE (overall increased persistence, but no difference between conditions) Conventional PREE rarely if ever observed in within-subjects manipulations of CRF - PRF

21 PREE as a generalization: Ecological validity If applied to a situation with a very specific schecule for a specific behavior  PREE Example: Single mother – child is begging for toys only from mom If applied to various situations with mixed contingencies  Reversed PREE Generalized PREE Example: Mother and father – child begs for toys from both

22 Response unit issue

23 Free operant responding: What is the response unit? Mowrer & Jones,1945: What should be counted as the response unit - single responses or the unit of responses required for reinforcement? Free-operant Intermittent reinforcemet, e.g., FR4

24 Response unit FR4 Reinforced responses

25 PREE Total responses Total responses / reinforcement ratio Reversed PREE

26 Nevin: PREE is an artefact

27 PREE: Alternative analyses Nevin, 1988: Behavioral momentum ”RPREE” is the rule – the response is stronger following CRF in free-operant responding (but not in discrete-trial experiments) following extended training Extinction performance Traditional measure: Number of responses Nevin: Slope of the extinction curve

28 PREE RPREE SHORTLONG Absolute number of responses Relative to initial ext response level Nevin, 1988

29 PREE vs. RPREE – important variables Dependent measure No. of responses vs. relative change Type of situations Free operant vs. discrete trial Complexity of situation One vs. more schedules (e.g., multiple schedule) Design Between groups vs. within subjects

30 PREE typically observed MeasureNumber of responses SituationDiscrete trial ScheduleSingle DesignBetween-groups manipulation of reinforcer rate OtherCRF schedule must be 100%

31 PREE: My interests Interaction PREE & Reversed PREE Cognition (verbalization) related to behavioral PREE

32 The experimental situation ”Computer responses” presented Left, right Subject responses recorded Left, right

33 The experimental situation Task Complete a four-response chain of responses started by the computer E.g.: Computer: L R Subject: R L Instructed task: Identify and apply the functional rule(s) ”Obtain as many correct answers as you can.” Rules (depending on experiment) ”Repeat computer sequence” ”Reverse computer sequence” Feedback (visual, autitory) for correct answer; nothing happens if answer is incorrect

34 The experimental situation Manipulations (between groups and/or within groups) Rule Reverse (typically used) Repeat Contingency CRF (100%) PRF (20-60%)

35 The experimental situation Reward rate manipulated Between groups Within subjects (multiple schedule) Discrete trial situation; fixed number of trials 180 acquisition trials 40 extinction trials

36 Conventinal PREE; operant responding; students; Svartdal, 2003, Exp. 4

37 Reversed & conventional PREE; operant responding; students; Svartdal, 2000 Reversed PREE Purpose: Explore the relationship between PREE and RPREE PREE vs. RPREE: Contradiction or compatible effects? Method Independent groups: PRF and CRF Within: CRF and PRF

38 Svartdal, 2000 ctd. Multiple schedule, alternating Group 40/40 Half trials (signalled): 40% Group 80/80 Half trials (signalled): 80% Group 80/40 Half trials (signalled): 80% Half trials (signalled): 40% PRF ”CRF” ”CRF” + PRF

39 PREE 80% 40% * No. of responses: RPREE * Relative change: No difference

40 Svartdal, 2000 ctd. Relationship between schedule components Simplest assumption: Modulation between component schedules: 60% + context = 60%  reference 60% + context = 100%  reduced persistence 60% + context = 20%  increaced persistence

41 Performance of a 60% schedule depending on other schedule = 100%, 60%, or 20% Svartdal, 2000

42 Svartdal, F. (2000). Persistence during extinction: Conventional and Reversed PREE under multiple schedules. Learning and Motivation, 31, 21-40.Persistence during extinction: Conventional and Reversed PREE under multiple schedules.

43 Cognition in PREE Currently: Strong cognitive arguments to interpret conditioning in terms of cognition Classical conditioning: Lovibond & Shanks, 2002 Operant conditioning: Shanks & St John, 1994 Implicit learning doubted: Shanks, 2005 Extinction: Lovibond, 2004 Basic argument: CONTINGENCY  CONSCIOUS APPREHENSION  BEHAVIORAL CHANGE CONTINGENCY  CONSCIOUS APPREHENSION  NO BEHAVIORAL CHANGE Large number of studies supporting this assumption

44 Cognition in PREE So, since the behvioral PREE is very robust, a ”cognitive PREE” must be easy to measure Basic prosedure: Behavioral acquisition under 100% vs. 60% reinforcer rate Measurement of verbalized PREE

45 Cognition in PREE Prediction of persistence: ”How likely is it that you will continue responding if reward no longer appears?” Several experiments have demonstrated no sensitivity to learning history in predictions

46 3 extinction trials; immediate behavioral sensitivity No difference in predictions Svartdal & Silvera, in prep.

47 Cognition in PREE Retrospective judgments: ”How many responses did you emit after reward no longer appeared?” Subjects are very accurate in descrbing their own behavior, including their own extinction persistence

48 Cognition in PREE Svartdal, F. (2003). Extinction after partial reinforcement: Predicted vs. judged persistence. Scandinavian Journal of Psychology, 44, 55-64.Extinction after partial reinforcement: Predicted vs. judged persistence.

49 Meta-cognitive PREE? We all have long experience with various contingencies Maybe a ”meta-cognition” evolves: Uncertain outcomes  Persist Certain outcomes  Quit

50 Meta-cognitive PREE? Scenarioes presented to subjects, manipulation Reliable outcome vs. Unreliable outcome Persistence judgments of behavior

51 Meta-cognitive PREE? Naive students: No effect of outcome manipulation

52 Meta-cognitive PREE? Psychology students (have read about PREE) Naive students

53 Meta-cognitive PREE? Svartdal, F. (2000). Persistence during extinction: Are judgments of persistence affected by contingency information? Scandinavian Journal of Psychology, 41, 315-328.

54 PREE: Theory Mowrer & Jones: Diskriminasjonshypo- tesen PRF: Læringbetingelsene  ekstinksjonsbetingelsene Generalisering til ekstinksjon CRF: Læringbetingelsene # ekstinksjonsbetingelsene Liten generalisering til ekstinksjon

55 PREE: Theory Amsel: Frustrasjonshypotesen PRF: Forventning om belønning  frustrasjon når belønning uteblir Frustrasjons-cues assosieres med læringssituasjonen Under ekstinksjon: Frustrasjon pga uteblitt belønning Læringssituasjonen  ekstinksjonssituasjonen CRF: Frustrasjon oppstår ikke under læring Læringssituasjonen # ekstinksjonssituasjonen

56 PREE: Theory Capaldi: Sequential hypothesis PRF: Ikke-belønnede trials blir signal på at belønning snart vil følge: … N N N R N N N R … Dvs.: Det opparbeides en forventning om belønning når belønning uteblir Under ekstinksjon: Mange responser pga forventning om belønning CRF: Ingen erfaring med uteblitt belønning under læring Under ekstinksjon: Få responser

57 PREE: Theory Status: Diskriminasjonshypotesen står svakt Amsels hypotese står rimelig sterkt Capaldis hypotese står ganske sterkt Nevins modell: Ingen hypotese i vanlig forstand Discrete-trial-situasjonen Capaldi og Amsel dominerende Fri-operant-situasjonen Svak teoretisk forståelse

Download ppt "The partial reinforcement extinction effect (PREE) Frode Svartdal University of Tromsø Oct. 2013."

Similar presentations

Ads by Google