1. Comparative Models of Classical Conditioning
Modern Twists on Rescorla-Wagner

2. Assumptions of Rescorla-Wagner (1974) model
Model developed to accurately predict and map learning as it occurs trial by trial
Assumes a bunch of givens:
Assume animal can perceive CS and US, and can exhibit UR and CR
Helpful for the animal to know 2 things about conditioning:
what TYPE of event is coming
the SIZE of the upcoming event
Thus, classical conditioning is really learning about:
signals (CS's) which are PREDICTORS for
important events (US's)

3. Assumptions of R-W model
Assumes that with each CS-US pairing 1 of 3 things can happen:
The CS might become more INHIBITORY
The CS might become more EXCITATORY
There is no change in the CS
How do these 3 rules work?
If US is larger than expected: CS = excitatory
If US is smaller than expected: CS= inhibitory
If US = expectations: No change in CS
The effect of reinforcers or nonreinforcers on the change of associative strength depends upon:
The existing associative strength of THAT CS
AND on the associative strength of other stimuli concurrently present

4. Explanation of how an animal anticipates what type of CS is coming:
Direct link is assumed between "CS center" and "US center":
E.g. between a tone center and food center
In 1970’s: other researchers thought R and W were crazy with this idea
Now: neuroscience shows formation of neural circuits!
Assumes that STRENGTH of an event is given
The conditioning situation is predicted by the strength of the learned connection
THUS: when learning is complete:
The strength of the association relates directly to the size or intensity of the CS
Asymptote of learning = max learning that can occur to that size or intensity of a CS
Maximum amount of learning that a given CS can support

5. More assumptions
The change in associative strength of a CS as the result of any given trial can be predicted from the composite strength resulting from all stimuli presented on that trial:
Composite strength = summation of conditioning that occurs to all stimuli present during a conditioning trial
If composite strength is LOW:
the ability of reinforcer to produce increments in the strength of component stimuli is HIGH
More can be learned for this trial
If the composite strength is HIGH:
reinforcement is relatively less effective (LOW)
Less can be learned for this trial- approaching max of learning

6. WHY is this equation important?
We can use the three rules to make predictions about amount and direction of classical conditioning
λ j > Vsum = Excitatory Conditioning
The degree to which the CS predicted the size of the US was GREATER than expected, so you react MORE to the CS next trial
λ j < Vsum = Inhibitory Conditioning
The degree to which the CS predicted the size of the US was LESS than expected, so you react LESS to the CS next trial
λ j = Vsum = no change:
The CS predicted the size of the US exactly as you expected

7. Can also explain why probability of reward given CS vs no CS makes a difference:
π = probability of US given the CS or No US given No CS
Are 3 basic rules:
If p(US|CS)>p(US|noCS): πax > πa
then Vx should be POSITIVE
If p(US|CS)<p(US|noCS): πax < πa
then Vx should be NEGATIVE
If p(US|CS)=p(US|noCS): πax = πa
then Vx should be ZERO
Vi = πaxαißj(Λj-Vsum)

8. Explaining loss of Associate Value despite pairings with the US:
Is an unusual prediction: Associative value of a CS can decrease despite continued pairings with the US
Show this with three-phase experiment:
Phase 1:
CSA  US (1 food pellet) and CSB US (I food pellet
on separate trials (equal #)
Phase 2: CSA and CSB paired together 1 food pellet
Same US, so same amount of conditioning, right??
RW model predicts that conditional properties of A and B individually will DECREASE in phase 2.

9. Explaining loss of Associate Value despite pairings with the US:
Why: Overexpectation
VA= λ ; VB= λ for phase 1;
VA+B = 2 λ
This is an over-expectation effect (we will see this in operant, too!)
Got 1 pellet with A
Got 1 pellet with B
So how many pellets when A+B?
Not get 2 pellets, so Phase 2 = decremental conditioning

10. Conditioned Inhibition
RW model also predicts Conditioned inhibition
Can test for Conditioned inhibition
2 kinds of trials
CS+US trials
CS-  no US trials
Consider 2 kinds of trials separately
Reinforced trials: CS+ gains excitatory properties
Unreinforced trials: CS- gains inhibitory properties
When presented together: cancel one another out (assuming equal conditioning)
Is a SUMMATIVE effect

11. Extinction of excitation and inhibition
During extinction:
EXT: CS+ no US
Initially: animal expects CS-US and is excitatory
This excitatory expectation is quickly diminished with repeated trials of no US
Decreases the excitatory conditioning incrementally, just like the excitatory conditioning was increased incrementally during learning

12. Dopamine and Rescorla Wagner Model
Turns out that changes in dopamine (DA) levels in dorsal striatal limbic cortical pathway vary as we learn
And guess what: these levels can be predicted by the RW model!
But, once a CS-US pairing (or an operant R-SR pairing) become well learned, the circuit begins to involve lower parts of the brain
Circuit begins to involve basal striatal areas
Becomes an “automated” or mastered behavior
No longer involves being “surprised”
Only returns to this pathway if the CS-US relation change!!!!!

13. Critique of the Rescorla-Wagner Model:
R-W model really a theory about the US effectiveness:
Says little about CS effectiveness
Doesn’t really deal with how CS effectiveness may vary
How WELL a CS predicts as a combo of salience and probability but doesn’t really evaluate this
States that an unpredicted US is effective in promoting learning, whereas a well-predicted US is ineffective because nothing is left to learn.

14. Critique of the Rescorla-Wagner Model:
Fails to predict the CS-pre-exposure effect:
Two groups of subjects
Grp I CS-US pairings Control
Grp II CS alone CS-US pairings PRE-Expos
Bob and Tom effect
Bob always hangs with Tom
You are dating Tom
You have a BAAAAAD breakup with Tom
Now you hate Bob….why?
Original model didn’t address how exposure could change salience/learning to CS
Incidental “CS” exposure (before it was a CS) can affect its salience and learn-ability
We learn about stimuli even if they aren’t predictive
We learn they AREN’T predictive
We learn “who they hang around”…..stimuli that group together

15. Critique of the Rescorla-Wagner Model:
In pre-exposure effect, simply being around a neutral stimulus alters its ability to become conditioned
Original R-W model doesn't predict any difference,
Assumes no conditioning trials occur when CSs presented in absence of US so Vsum = 0
This appears to be wrong
Conditioning likely occurring any time 2 stimuli are together
Form an incidental association
Need to modify the equation to account for this
They have, but we won’t!

16. Critique of the Rescorla-Wagner Model:
Original R-W model implies that salience is fixed for any given CS
R-W assume CS salience doesn't change w/experience
More recent data strongly suggest CS salience DOES change w/experience
Research shows CS salience CHANGES!
Why changes in CS salience?
Data suggest that Salience to a CS DECREASES when CS is repeatedly presented without consequence
CS that is accidentally paired with another CS INCREASES in salience
Thus: Appears that CS and US effectiveness are both highly important
As a result of these types of data: Given birth to attentional models of CC

17. Three Primary Extensions and Alternatives to RW model
Attentional models
Timing and Information models
Comparator hypothesis

18. Attentional Models of CC
Question: How well does the CS command attention?
Assumes that increased attention facilitates learning about a stimulus
Procedures that disrupt attention to CS disrupt learning
Different attentional models differ in assumptions about what determines how much attention a CS commands on any given trial
Single attentional mechanisms: Kamin’s surprise
Multiple attentional mechanisms: Current models examine changes in multiple types of attention

19. Multiple attentional mechanisms:
Three attentions:
Looking for action: attention a CS commands after it has become a good predictor of the CS
Looking for learning: how well the organism processes cues that are not yet good predictors of the US, and thus have to be “learned about”
Looking for liking: the emotional/affective properties of the CS
Assume that the outcome of a given trial alters the degree of attention commanded by the CS on future trials
Surprise? Then an increase in looking for learning on next trial
Good predictor? Then increase in attention on next trial
Pleasant outcome? Increases emotional value of CS on next trial

20. Timing/Information Theory Models
Recognized that time is important factor in CC: Timberlake, et al 1980
Focal search
Responses become conditioned when CS-US interval is short
Short CS-US intervals condition responses congruent with a focal search
Immediate contact with the CS; food preparation responses
General search
Different responses become conditioned when CS-US interval is long
Long CS-US intervals condition responses congruent with a general search
Longer intervals result in general search responses such as looking, searching, etc.
Suggests that organisms learn both
WHAT is predicted
HOW MUCH is predicted
WHEN what is predicted will occur.

21. Temporal coding hypothesis
Organisms learn when the US occurs in relation to the CS
Use this information in blocking, second-order conditioning, etc.
Not pay attention to the tone, because it occurs with the light
If the tone were to occur before/after the light, then different CS situation
What is learned in one phase of training influences what is learned in subsequent phase
Overshadowing
Pre-exposure effect
Blocking
Large literature supports this

22. Importance of Inter-trial interval
More conditioned responding observed with a longer inter-trial interval
Inter-trial interval is the time BETWEEN trials
If very short; less learning
If longer; more learning
BUT: Intertrial interval and CS duration (CS-US interval) act in combination to determine responding
Critical factor: Relative duration of these two temporal intervals rather than absolute value of either one by itself
Need quick CS-US pairing (delayed conditioning) then a longer inter-trial interval
Why?

23. Importance of Inter-trial interval
Holland (2000)
Conditioned rats to an auditory cue that was presented just before delivery to food
CR to CS: Nosing of food cup (goal tracking)
Each group conditioned with:
1 of 2 CS durations: 10 or 20 sec
1 of 6 intertrial intervals: 15 to 960 sec
Characterized responses in terms of the ratio of the intertrial interval (I) and the CS duration (T).
Time spent nosing the food cup during CS plotted as function of relative value of I/T
Results: as I/T ratio increases, the percentage of time the rats spend with the nose in the food cup increases

24. Why is the Inter-trial interval so important?
Relative Waiting Time Hypothesis
Organism is making a comparison between events during the Intertrial interval (I) and Trial interval (T)
How long one has to wait for the US during the CS vs.
How long one has to wait for the US during the intertrial interval
When US waiting time during CS is shorter than intertrial interval:
I/T ratio is high
CS is highly informative about the next occurrence of the US
Lots of responding
When US waiting time during CS is same/longer than intertrial interval wait:
I/T ratio is low
CS is not highly informative
Less responding

25. Comparitor Hypothesis
Ralph Miller, et al.: Organism compares across learning situations
Assumes: Conditional responding depends on
What happens during CS
ALSO what happens in other aspects of experimental situation (when CS is not presented)
That is, animal compares presence of CS to absence of CS
Revaluation effects: Can better explain blocking
What is blocked is RESPONDING to CSAB, not learning of CSAB
Can unmask blocking to CSAB by presenting CSA alone without US (EXT CSA).
Changes conditional value of CSA
Now CSAB has different, predictive value
Animal “revalues” CSAB and now responds as it is predictive, whereas CSA is not.

26. Comparator Hypothesis
Note: This is a theory of PERFORMANCE, not learning!
Assumes conditioned responding depends on BOTH associations between CS-US and between US and other stimuli when CS not there
These other stimuli form the “comparator” cues
Also assumes formation of excitatory associations with US ONLY
Instead of forming inhibitory associations with CS, form stronger excitatory associations with “not CS”
Animal compares the relative associative value of “CS vs. “not CS”
Whatever is stronger is what you respond to

27. Why are these theories important?
Helps explain the PROCESS of classical conditioning
Describes conditions and allows predictions of
Who is likely to form associations between stimuli
To What kinds of stimuli it will occur
When classical conditioning will occur
Where associations form: what settings/conditions are important
How well: how fast/slow, what quality of associations are formed.
Important for predicting/controlling behavior in applied settings
Understanding and treatment of pedophilia, sexual fetishes, etc.
Phobia treatment
Commercials and advertising
Social behaviors