1. Chapter 6: Learning

2. Classical Conditioning
Ivan Pavlov
Terminology
Unconditioned Stimulus (UCS)
Conditioned Stimulus (CS)
Unconditioned Response (UCR)
Conditioned Response (CR)
Classical conditioning explains how a neutral stimulus can acquire the capacity to elicit (or draw forth) a response originally elicited by another stimulus.
Ivan Pavlov, a prominent Russian physiologist in the early 1900s, who did Nobel prize winning research on digestion, discovered (partly by accident) that dogs will salivate in response to the sound of a tone. In doing so, he discovered classical, sometimes called Pavlovian, conditioning.
In classical conditioning, the UCS is a stimulus that elicits an unconditioned response without previous conditioning: Pavlov’s meat powder.
The UCR is an unlearned reaction to a UCS that occurs without previous conditioning: salivating.
The CS is a previously neutral stimulus that has acquired the capacity to elicit a conditioned response: the sound of a tone.
The CR is a learned reaction to a conditioned stimulus: salivating to the tone.

3. Figure 6.1 Classical conditioning apparatus
Classical conditioning apparatus. An experimental arrangement similar to the one depicted here
(taken from Yerkes & Morgulis, 1909) has typically been used in demonstrations of classical
conditioning, although Pavlov’s original setup (see inset) was quite a bit simpler. The dog is restrained
in a harness. A tone is used as the conditioned stimulus (CS), and the presentation of meat powder is used as
the unconditioned stimulus (UCS). The tube inserted into the dog’s salivary gland allows precise measurement
of its salivation response. The pen and rotating drum of paper on the left are used to maintain a continuous
record of salivary flow. (Inset) The less elaborate setup that Pavlov originally used to collect saliva on
each trial is shown here (Goodwin, 1991).
Figure 6.1 Classical conditioning apparatus

4. Figure 6.2 The sequence of events in classical conditioning
(a) Moving downward,
this series of three panels
outlines the sequence of
events in classical conditioning,
using Pavlov’s
original demonstration as
an example. (b) As we encounter
other examples
of classical conditioning
throughout the book, we
will see many diagrams like
the one in this panel, which
will provide snapshots of
specific instances of classical
conditioning.
Figure 6.2 The sequence of events in classical conditioning

5. Figure 6.3 Classical conditioning of a fear response
Classical conditioning of a fear response. Many emotional responses that would otherwise be puzzling can be explained by classical conditioning. In the case of one woman’s bridge phobia, the fear originally elicited by her father’s scare tactics became a conditioned response to the
stimulus of bridges.
Figure 6.3 Classical conditioning of a fear response

6. Basic Processes in Classical Conditioning
Acquisition
Extinction
Spontaneous recovery
Stimulus generalization
Stimulus discrimination
Higher-order conditioning
Acquisition refers to the initial stage of learning a response: acquiring the response.
Extinction occurs when the CS and UCS are no longer paired and the response to the CS is weakened. We know that the response is still there, just not active, because of spontaneous recovery – when an extinguished response reappears after a period of non-pairing.
Generalization occurs when conditioning generalizes to additional stimuli that are similar to the CS; for example, Watson and Rayner’s study with Little Albert, who was conditioned to fear a white rat but later came to be afraid of many white, furry objects.
Discrimination is the opposite of generalization; that is, the response is to a specific stimulus. Similar stimuli don’t work.
Higher order conditioning occurs when a CS functions as if it were a UCS to establish new conditioning.

7. Figure 6.6 Acquisition, extinction, and spontaneous recovery
Classical conditioning
and romance. Pleasant
emotional responses can be
acquired through classical
conditioning, as illustrated
by one woman’s unusual
conditioned response to
the aroma of Beemans gum
and cigarette smoke.
Figure 6.6 Acquisition, extinction, and spontaneous recovery

8. Figure 6.8 Higher-order conditioning
involves a twophase
process. In the first
phase, a neutral stimulus
(such as a tone) is paired
with an unconditioned
stimulus (such as meat
powder) until it becomes a
conditioned stimulus that
elicits the response originally
evoked by the UCS
(such as salivation). In the
second phase, another neutral
stimulus (such as a red
light) is paired with the
previously established CS,
so that it also acquires
the capacity to elicit the
response originally evoked
by the UCS.
Figure 6.8 Higher-order conditioning

9. B.F. Skinner (1953) – principle of reinforcement “Skinner box”
Operant Conditioning
B.F. Skinner (1953) – principle of reinforcement
“Skinner box”
Emission of response
Reinforcement contingencies
Cumulative recorder
Skinner’s principle of reinforcement holds that organisms tend to repeat those responses that are followed by favorable consequences, or reinforcement.
Skinner defined reinforcement as when an event following a response increases an organism’s tendency to make that response.
Skinner created a prototype experimental procedure, using animals and a “Skinner box.” This is a small enclosure in which an animal can make a specific response that is recorded, while the consequences of the response are systematically controlled. Rats, for example, press a lever.
Because operant responses tend to be voluntary, they are said to be emitted rather than elicited.
Reinforcement contingencies are the circumstances, or rules, that determine whether responses lead to the presentation of reinforcers.
The cumulative recorder creates a graphic record of responding and reinforcement in a Skinner box as a function of time.

10. Figure 6.9 Reinforcement in operant conditioning
Reinforcement in operant conditioning. According to Skinner, reinforcement occurs
when a response is followed by rewarding consequences and the organism’s tendency to
make the response increases. The two examples diagrammed here illustrate the basic
premise of operant conditioning—that voluntary behavior is controlled by its consequences.
These examples involve positive reinforcement (for a comparison of positive and negative
reinforcement, see Figure 6.14).
Figure 6.9 Reinforcement in operant conditioning

11. Figure 6.10 Skinner box and cumulative recorder
Skinner box and cumulative recorder. (a) This diagram highlights some of the key features of
an operant chamber, or Skinner box. In this apparatus designed for rats, the response under study is
lever pressing. Food pellets, which may serve as reinforcers, are delivered into the food cup on the
right. The speaker and light permit manipulations of visual and auditory stimuli, and the electric grid
gives the experimenter control over aversive consequences (shock) in the box. (b) A cumulative
recorder connected to the box keeps a continuous record of responses and reinforcements. A small
segment of a cumulative record is shown here. The entire process is automatic as the paper moves
with the passage of time; each lever press moves the pen up a step, and each reinforcement is marked
with a slash. (c) This photo shows the real thing—a rat
being conditioned in a Skinner box. Note the food dispenser
on the left, which was omitted from the diagram.
Figure Skinner box and cumulative recorder

12. Basic Processes in Operant Conditioning
Acquisition- the formation of a conditioned response tendency.
Shaping- the reinforcement of closer and closer approximations of a desired response.
Necessary when an organism does not emit a desired behavior on its own.
Extinction- the gradual weakening and disappearance of a response tendency because the response is no longer followed by reinforcement.
As in classical conditioning, acquisition refers to the initial stage of learning.
Learning operant responses usually occurs through a gradual process called shaping, which consists of the reinforcement of closer and closer approximations of a desired response. This is the key in pet tricks.
Extinction in operant conditioning refers to the gradual weakening and disappearance of a response tendency, because the response is no longer followed by a reinforcer. If we stop giving food when the rat presses the lever, this results in a brief surge of responding followed by a gradual decline until it approaches zero.
Stimuli that precede a response can exert considerable influence over operant behavior, basically becoming “signals” that a reinforcer is coming. Discriminative stimuli are cues that influence operant behavior by indicating the probable consequences of a response (ex. slow down when the highway is wet, ask Mom when she’s in a good mood, etc.).
Discrimination occurs when an organism responds to one stimulus, but not another one similar to it, while generalization occurs when a new stimulus is responded to as if it were the original. (Ex. cat runs to the sound of a can-opener which signals food, but not to the sound of the mixer: discrimination. Get a new blender, cat runs to it: generalization).

13. Stimulus Control
Discriminative Stimuli- cues that influence operant behavior by indicating the probable consequences of a response.
ex. children learn to ask for sweets when their parents are in a good mood.
Discrimination- occurs when an organism responds to one stimulus, but not another one similar to it.
ex. cat runs to the sound of a can-opener which signals food, but not to the sound of the mixer
Generalization occurs when a new stimulus is responded to as if it were the original.
ex. : Get a new blender, cat runs to it:

14. Table 6.1 Comparison of Basic Processes in Classical and Operant Conditioning

15. Reinforcement: Consequences that Strengthen Responses
Primary Reinforcers
Satisfy biological needs (food, water, warmth, sex)
Inherently reinforcing
Limited because they are closely tied to physiological needs.
Secondary Reinforcers
Conditioned reinforcement (money, good grades, attention)
Develop through learning
Vary among members of a species
Operant theorists distinguish between primary reinforcers, which are events that are inherently reinforcing because they satisfy biological needs, and secondary reinforcers, which are events that acquire reinforcing qualities by being associated with primary reinforcers.
Primary reinforcers in humans include food, water, warmth, sex, and maybe affection expressed through hugging and close bodily contact.
Secondary reinforcers in humans include things like money, good grades, attention, flattery, praise, and applause.

16. Schedules of Reinforcement
Continuous reinforcement-occurs when every instance of a designed response is reinforced.
Intermittent (partial) reinforcement-occurs when a response is reinforced only some of the time.
Ratio schedules- tend to yield higher response rates
Fixed
Variable
Interval schedules- tend to yield more resistance to extinction
A schedule of reinforcement determines which occurrences of a specific response result in the presentation of a reinforcer.
Continuous reinforcement occurs when every instance of a designated response is reinforced (faster acquisition, faster extinction).
Intermittent reinforcement occurs when a designated response is reinforced only some of the time (greater resistance to extinction).
Ratio schedules require the organism to make the designated response a certain number of times to gain each reinforcer. A fixed-ratio schedule entails giving a reinforcer after a fixed number of non-reinforced responses. A variable ratio schedule entails giving a reinforcer after a variable number of non-reinforced responses.
Interval schedules require a time period to pass between the presentation of reinforcers. A fixed-interval schedule entails reinforcing the first response that occurs after a fixed time interval has elapsed. A variable-interval schedule entails giving the reinforcer for the first response after a variable time interval has elapsed.
More than 40 years of research on these schedules has yielded an enormous amount of information about how organisms respond to different schedules.

17. Figure 6.13 Schedules of reinforcement and patterns of response
Schedules of reinforcement and patterns of response. In graphs of operant responding such as these, a steeper slope
indicates a faster rate of response and the slash marks reflect the delivery of reinforcers. Each type of reinforcement schedule
tends to generate a characteristic pattern of responding. In general, ratio schedules tend to produce more rapid responding
than interval schedules (note the steep slopes of the FR and VR curves). In comparison to fixed schedules, variable schedules
tend to yield steadier responding (note the smoother lines for the VR and VI schedules on the right).
Figure Schedules of reinforcement and patterns of response

18. Consequences: Reinforcement
Increasing a response:
Positive reinforcement = response followed by rewarding stimulus
Negative reinforcement = response followed by removal of an aversive stimulus
Escape learning- an organism acquires a response that decreases or ends some aversive stimulation
Avoidance learning- an organism acquires a response that prevents some aversive stimulation from occurring
Decreasing a response:
Punishment
Problems with punishment
Responses can be strengthened either by presenting positive reinforcers or by removing negative reinforcers.
Negative reinforcement regulates escape and avoidance learning. In escape learning, an organism learns to perform a behavior that decreases or ends aversive stimulation (turning on the air conditioner). In avoidance learning, an organism learns to prevent or avoid some aversive stimulation (turn on the a/c before it gets too hot).
Punishment occurs when an event following a response weakens the tendency to make that response. Punishment is much more than disciplinary procedures. You wear a new outfit and friends laugh—that’s punishing. Punishment may involve presentation of an aversive stimulus (spanking) or removal of a rewarding stimulus (taking away TV).
Some of the problems associated with punishment are that it can trigger strong emotional responses (anxiety, anger, resentment, hostility); physical punishment can lead to an increase in aggressive behavior.

19. Figure 6.15 Escape and avoidance learning
Escape and avoidance learning. (a) Escape and avoidance learning are often studied with a shuttle box like that shown here. Warning signals, shock, and the animal’s ability to flee from one compartment to another can be controlled by the experimenter. (b) Avoidance begins because classical
conditioning creates a conditioned fear that is elicited by the warning signal (panel 1). Avoidance continues because it is maintained by operant conditioning (panel 2). Specifically, the avoidance response is strengthened through negative reinforcement, since it leads to removal of the conditioned fear.
Figure 6.15
Escape and avoidance learning. (a) Escape and avoidance
learning are often studied with a shuttle box like that
shown here. Warning signals, shock, and the animal’s ability
to flee from one compartment to another can be controlled
by the experimenter. (b) Avoidance begins because classical
conditioning creates a conditioned fear that is elicited by
the warning signal (panel 1). Avoidance continues because it
is maintained by operant conditioning (panel 2). Specifically,
the avoidance response is strengthened through negative reinforcement,
since it leads to removal of the conditioned fear.
Figure Escape and avoidance learning

20. Figure 6.14 Positive reinforcement versus negative reinforcement
In positive
reinforcement, a response
leads to the presentation
of a rewarding stimulus. In
negative reinforcement, a response
leads to the removal
of an aversive stimulus.
Both types of reinforcement
involve favorable consequences
and both have the
same effect on behavior:
The organism’s tendency
to emit the reinforced
response is strengthened.
Figure Positive reinforcement versus negative reinforcement

21. Consequences Punishment: Decreasing a response
Punishment- occurs when an event following a response weakens the tendency to make that response.
Problems with punishment- may result in side effects such as negative emotional responses and increased aggressive behavior.
When used for disciplinary reasons, punishment should be: applied swiftly, just severe enough to be effective, explained and not physical

22. Figure 6.16 Comparison of negative reinforcement and punishment
punishment. Although
punishment can occur
when a response leads to
the removal of a rewarding
stimulus, it more typically
involves the presentation
of an aversive stimulus.
Students often confuse
punishment with negative
reinforcement because they
associate both with aversive
stimuli. However, as this
diagram shows, punishment
and negative reinforcement
represent opposite procedures
that have opposite
effects on behavior.
Figure Comparison of negative reinforcement and punishment

23. Biological Constraints on Conditioning
Conditioned Taste Aversion- discovered that visual and auditory stimuli followed by nausea failed to produce conditioned aversions, but taste-nausea associations are impossible to prevent. John Garcia argued that the taste-nausea connection is a by-product of the evolutionary history of mammals. Ex. Animals that consume poisonous foods and survive must learn not to repeat their mistakes.
New research has greatly changed the way we think about conditioning, with both biological and cognitive influences having been discovered.
Instinctive drift occurs when an animal’s innate response tendencies interfere with conditioning (the raccoon who would rather rub the coins together than obtain the reinforcer).
Conditioned taste aversions can be readily acquired, after only one trial and when the stimuli are not contiguous (i.e., becoming ill occurs hours after eating a food), suggesting that there is a biological mechanism at work.
While research has traditionally focused on conditioning with arbitrary stimuli, Michael Domjan points out that conditioning in natural settings does not typically involve arbitrary stimuli, but stimuli that have a natural relationship to the unconditioned stimuli they predict.
Edward C. Tolman showed that rats learned a maze even in the absence of reinforcement (what he called latent learning), challenging the assumption that learning was dependent on reinforcement, and argued that the rats had formed a cognitive map of the maze.
Signal relations theory (Rescorla) illustrates that the predictive value of a CS is an influential factor governing classical conditioning.
Response-outcome relations - when a response is followed by a desired outcome, it is more easily strengthened if it seems that it caused the outcome (predicts). You study for an exam and listen to Smashmouth, you make an A. What is strengthened, studying or listening to Smashmouth?
Signal relations and response-outcome research suggest that cognitive processes play a larger role in conditioning than once believed.

24. Arbitrary vs. Ecological Conditioned Stimuli
Michael Domjan argued that rapid learning seen in conditioned taste aversion is not all that unique.
Previous studies of classical conditioning have always paired the unconditioned stimulus with a neutral stimulus that was unrelated.
The association that was being created was not a product of previous learning but rather a newly acquired association.
Argued that researchers should shift their focus from arbitrary , neutral stimuli to ecologically relevant conditioned stimuli

25. Cognitive Influences on Conditioning
Latent Learning- learning that is not apparent from behavior when it first occurs.
ex. rat in group C improving and surpassing the performance of the rat in group A after being rewarded on the 11th try.
Signal Relations- environmental stimuli serve as signals and some are better/dependable than others.
A ‘good’ signal is one that allows accurate prediction of the US.
Suggests that classical conditioning may involve information processing rather than reflexive responding.
Response-Outcome Relations and Reinforcement- when a response is followed by a desirable outcome, the response is more likely to be strengthened if it appears to have caused the favorable outcome.

26. Figure 6.19 Latent learning
Latent learning. (a) In
the study by Tolman and
Honzik (1930), rats learned
to run the complicated
maze shown here. (b) The
results obtained by Tolman
and Honzik (1930) are
summarized in this graph.
The rats in Group C
showed a sudden improvement
in performance when
a food reward was introduced
on Trial 11. Tolman
concluded that the rats in
this group were learning
about the maze all along,
but their learning remained
“latent” until reinforcement
was made available.
Figure Latent learning

27. Observational Learning
Albert Bandura
Observational learning- occurs when an organism’s responding is influenced by the observation of others, who are called models.
Learning through observation accounts for a great deal of learning in both animals and humans.
Stated that classical and operant conditioning can take place vicariously through observational learning
Can explain why exposure to media violence correlates with increased aggression.
Albert Bandura outlined the theory of observational learning. In observational learning, vicarious conditioning occurs by an organism watching another organism (a model) be conditioned. Observational learning can occur for both classical and operant conditioning.
In order for observational learning to take place, four key processes are at work. First the organism must pay attention to the model, retain the information observed, and be able to reproduce the behavior. Finally, an observed response is unlikely to be reproduced unless the organism is motivated to do so, i.e., they believe there will be a pay off.

28. Basic Processes of Observational Learning
Attention- to learn through observation, you must pay attention to another person’s behavior and its consequences.
Retention- you must store a mental representation of what you have witnessed in your memory.
Reproduction- enacting a modeled response by converting your stored mental images into overt behavior
Motivation- you are unlikely to reproduce an observed response unless you are motivated to do so.

29. Figure 6.20 Observational learning
In observational learning,
an observer attends to and
stores a mental representation
of a model’s behavior
(example: assertive bargaining)
and its consequences
(example: a good buy on a
car). If the observer sees the
modeled response lead to a
favorable outcome, the
observer’s tendency to emit
the modeled response will
be strengthened.
Figure Observational learning