2Classical conditioning Learningrelatively durable change in an organism’s behavior due to experienceIvan PavlovTerminologyUnconditioned Stimulus (UCS)Conditioned Stimulus (CS)Unconditioned Response (UCR)Conditioned Response (CR)Classical conditioning explains how a neutral stimulus can acquire the capacity to elicit (or draw forth) a response originally elicited by another stimulus.Ivan Pavlov, a prominent Russian physiologist in the early 1900’s, who did Nobel prize winning research on digestion, discovered (partly by accident) that dogs will salivate in response to the sound of a tone. In doing so, he discovered classical, sometimes called Pavlovian, conditioning.In classical conditioning, the UCS is a stimulus that elicits an unconditioned response without previous conditioning…Pavlov’s meat powder.The UCR is an unlearned reaction to a UCS that occurs without previous conditioning…salivating.The CS is a previously neutral stimulus that has acquired the capacity to elicit a conditioned response...the sound of a tone.The CR is a learned reaction to a conditioned stimulus…salivating to the tone.
9Classical Conditioning: More Terminology Trial = pairing of UCS and CSAcquisition = initial stage in learningStimulus contiguity = occurring together in time and space3 types of Classical ConditioningSimultaneous conditioning: CS and UCS begin and end togetherShort-delayed conditioning: CS begins just before the UCS, end togetherTrace conditioning: CS begins and ends before UCS is presentedIn classical conditioning research, a trial is a pairing of the UCS and the CS. (How many times have the tone and the meat powder been paired?) Some behaviors are learned after only one trial or pairing, while others take many trials.Acquisition refers to the initial stage of learning a response…acquiring the response.Conditioning has been shown to depend on stimulus contiguity; that is, the occurring of stimuli together in time and space.So when do you sound the tone in a classical conditioning task? What works best? Of the three types of conditioning (simultaneous, short-delayed, and trace), short-delayed conditioning appears to most promote acquisition of a classically conditioned response…ideally the delay should be very brief, about ½ a second.
10Processes in Classical Conditioning ExtinctionSpontaneous RecoveryStimulus GeneralizationDiscriminationHigher-order conditioningExtinction occurs when the CS and UCS are no longer paired and the response to the CS is weakened. We know that the response is still there, just not active, because of spontaneous recovery – when an extinguished response reappears after a period of non-pairing.Generalization occurs when conditioning generalizes to additional stimuli that are similar to the CS; for example, Watson and Rayner’s study with Little Albert, who was conditioned to fear a white rat but later came to be afraid of many white, furry objects.Discrimination is the opposite of generalization; that is, the response is to a specific stimulus… similar stimuli don’t work.Higher order conditioning occurs when a CS functions as if it were a UCS to establish new conditioning…condition to respond to a tone with saliva, pair the tone with a light.
13Behaviorism John B. Watson viewed psychology as objective science generally agreed-upon consensus todayrecommended study of behavior without reference to unobservable mental processesnot universally accepted by all schools of thought today
14Watson took a a baby named Albert and conditioned him to be afraid of white furry objects using Pavlov’s techniques.DiscPsy Fig5.2a p170Watson & Raynor with Little Albert
18Operant Conditioning or Instrumental Learning Edward L. Thorndike (1913) – the law of effectB.F. Skinner (1953) – principle of reinforcementOperant chamber (Skinner Box)Emission of responseReinforcement contingenciesCumulative recorderThorndike’s law of effect stated that if a response in the presence of a stimulus leads to satisfying effects, the association between the stimulus and the response is strengthened.This law became the cornerstone of Skinner’s theory.Skinner’s principle of reinforcement holds that organisms tend to repeat those responses that are followed by favorable consequences, or reinforcement.Skinner defined reinforcement as when an event following a response increases an organism’s tendency to make that response.Skinner created a prototype experimental procedure, using animals and an operant chamber or “Skinner box.” This is a small enclosure in which an animal can make a specific response that is recorded, while the consequences of the response are systematically controlled. Rats, for example, press a lever.Because operant responses tend to be voluntary, they are said to be emitted rather than elicited.Reinforcement contingencies are the circumstances, or rules, that determine whether responses lead to the presentation of reinforcers.The cumulative recorder creates a graphic record of responding and reinforcement in a Skinner box as a function of time.
21Basic Processes in Operant Conditioning AcquisitionShapingExtinctionSpontaneous RecoveryStimulus ControlGeneralizationDiscriminationAs in classical conditioning, acquisition refers to the initial stage of learning.Learning operant responses usually occurs through a gradual process called shaping, which consists of the reinforcement of closer and closer approximations of a desired response…key in pet tricks.Extinction in operant conditioning refers to the gradual weakening and disappearance of a response tendency, because the response is no longer followed by a reinforcer…stop giving food when the rat presses the lever…results in a brief surge of responding followed by a gradual decline until it approaches zero.Stimuli that precede a response can exert considerable influence over operant behavior, basically becoming “signals” that a reinforcer is coming (see Figure 6.17 re: roborats demonstrating stimulus control). Discriminative stimuli are cues that influence operant behavior by indicating the probable consequences of a response (ex. slow down when the highway is wet, ask Mom when she’s in a good mood, etc.).Discrimination occurs when an organism responds to one stimulus, but not another one similar to it, while generalization occurs when a new stimulus is responded to as if it were the original. (ex. cat runs to the sound of a can-opener which signals food, but not to the sound of the mixer…discrimination…get a new blender, cat runs to it...generalization).
24Reinforcement: Consequences that Strengthen Responses Delayed ReinforcementLonger delay, slower conditioningPrimary ReinforcersSatisfy biological needsSecondary ReinforcersConditioned reinforcementSkinner said that reinforcement occurs whenever an outcome strengthens a response…this definition avoids the use of “experience of pleasure,” which is an unobservable feeling.In operant conditioning, immediate reinforcement produces the fastest conditioning.Operant theorists distinguish between primary reinforcers, which are events that are inherently reinforcing because they satisfy biological needs, and secondary reinforcers, which are events that acquire reinforcing qualities by being associated with primary reinforcers.Primary reinforcers in humans include food, water, warmth, sex, and maybe affection expressed through hugging and close bodily contact.Secondary reinforcers in humans include things like money, good grades, attention, flattery, praise, and applause.
25Schedules of Reinforcement Continuous reinforcement – faster extinctionIntermittent (partial) reinforcement – greater resistance to extinctionRatio schedules (number)Fixed (set)Variable (random)Interval schedules (time)FixedVariableA schedule of reinforcement determines which occurrences of a specific response result in the presentation of a reinforcer.Continuous reinforcement occurs when every instance of a designated response is reinforced (faster acquisition, faster extinction).Intermittent reinforcement occurs when a designated response is reinforced only some of the time (greater resistance to extinction).Ratio schedules require the organism to make the designated response a certain number of times to gain each reinforcer. A fixed-ratio schedule entails giving a reinforcer after a fixed number of non-reinforced responses. A variable ratio schedule entails giving a reinforcer after a variable number of non-reinforced responses.Interval schedules require a time period to pass between the presentation of reinforcers. A fixed-interval schedule entails reinforcing the first response that occurs after a fixed time interval has elapsed. A variable-interval schedule entails giving the reinforcer for the first response after a variable time interval has elapsed.More than 50 years of research on these schedules has yielded an enormous amount of information about how organisms respond to different schedules.Figure 6.19, presented on the next slide, depicts some of what has been discovered.
27Consequences: Reinforcement and Punishment Increasing a response:Positive (+) reinforcement = response followed by rewarding stimulus (Give them something they want, ie a candy bar)Negative (-) reinforcement = response followed by removal of an aversive stimulus (Take away something that is annoying/bothering/hurting them, ie nagging, seat belt tone)Escape learning (open umbrella after you’re wet)Avoidance learning (open umbrella before it rains)Decreasing a response:Positive Punishment = presentation of an aversive stimulus (give them something they do not want, ie detention, fineNegative Punishment = removal of a rewarding stimulus (take away something they want, ie cell phone, carResponses can be strengthened either by presenting positive reinforcers or by removing negative reinforcers.Negative reinforcement regulates escape and avoidance learning. In escape learning, an organism learns to perform a behavior that decreases or ends aversive stimulation (turning on the air conditioner). In avoidance learning, an organism learns to prevent or avoid some aversive stimulation (turn on the a/c before it gets too hot).Punishment occurs when an event following a response weakens the tendency to make that response. Punishment is much more than disciplinary procedures…wear a new outfit and friends laugh…punishing. Punishment may involve presentation of an aversive stimulus (spanking) or removal of a rewarding stimulus (taking away TV).Some of the problems associated with punishment are that it can trigger strong emotional responses (anxiety, anger, resentment, hostility); physical punishment can lead to an increase in aggressive behavior.
32Problems with Punishment - (Spanking) Does not teach or promote alternative, acceptable behaviorMay produce undesirable results such as hostility, passivity, fearLikely to be temporaryMay model aggression (Bandura Bobo doll experiment)
34Biological Predispositions John GarciaConditioned taste aversionsNot all neutral stimuli can become conditioned stimuli.Internal stimuli—associate better with tasteExternal stimuli—associate better with painBiological preparedness
35Changes in Our Understanding of Conditioning Biological Constraints on ConditioningInstinctive DriftConditioned Taste AversionPreparedness and PhobiasCognitive Influences on ConditioningSignal relations – predicive value based on experienceResponse-outcome relations – causal relationshipCognitive processes play a large role in conditioningNew research has greatly changed the way we think about conditioning, with both biological and cognitive influences having been discovered.Instinctive drift occurs when an animal’s innate response tendencies interfere with conditioning (the raccoon who would rather rub the coins together than obtain the reinforcer).Conditioned taste aversions can be readily acquired, after only one trial and when the stimuli are not contiguous (i.e. becoming ill occurs hours after eating a food), suggesting that there is a biological mechanism at work.Martin Seligman has outlined the fact that some phobias are more easily conditioned than others, suggesting the concept of preparedness…that we are biologically prepared to learn to fear objects or events that have inherent danger.Signal relations theory (Rescorla) illustrates that the predictive value of a CS is an influential factor governing classical conditioning.Response-outcome relations - when a response is followed by a desired outcome, it is more easily strengthened if it seems that it caused the outcome (predicts)…you study for an exam and listen to Smash Mouth…you make an A. What is strengthened, studying or listening to Smash Mouth?Signal relations and response-outcome research suggest that cognitive processes play a larger role in conditioning than once believed.The evolutionary perspective on learning assumes that an organism’s biological heritage places certain constraints on the learning process (which some theorists see as merely specialized mechanisms designed to solve particular types of adaptive problems for particular species).
36Signal RelationsRobert Rescorla conducted research around the cognitive element in conditioningEnvironmental stimuli serve as signalsSLIDE 8: Signal Relations1) Robert Rescorla conducted research around the cognitive element in conditioningThe cognitive element in conditioning is especially prominent in research conducted by Robert Rescorla.2) Environmental stimuli serve as signalsRescorla claims that environmental stimuli serve as signals3) Some stimuli are better and more dependable signals than othersAnd that some stimuli are better, or more dependable, signals than others.4) Rescorla manipulated signal relations in classical conditioningThus, Rescorla manipulated signal relations in classical conditioning.Some stimuli are better and more dependable signals than othersRescorla manipulated signal relations in classical conditioning
37Signal Relations (ii)CS-UCS relations influence whether a CS is a good signalA ‘good’ signal is one that follows accurate prediction of the UCSSLIDE 9: Signal Relations (ii)1) CS-UCS relations influence whether a CS is a good signal.That is, CS-UCS relations that influence whether a CS is a good signal.2) A ‘good’ signal is one that follows accurate prediction of the UCSA ‘good’ signal is one that follows accurate prediction of the UCS.3) Example:Look at the following example.4) For some rats the CS (tone) and UCS (shock) are paired in 100% of the experimental trialsA tone and shock are paired 20 times for one group of rats. Otherwise, these rats are never shocked. For these rats the CS (tone) and UCS (shock) are paired in 100% of the experimental trials.Another group of rats also receive 20 pairings of the tone and shock. However, the rats in this group are also exposed to the shock on 20 other trials when the tone does not precede it.5) For another group the CS and UCS are paired in only 50% of the trialsFor this group, the CS and UCS are paired in only 50% of the trials.Example:For one group of rats the CS (tone) and UCS (shock) are paired in 100% of the experimental trialsFor another group the CS and UCS are paired in only 50% of the trials
38The predictive value of CS The two groups of rats have had an equal number of CS-UCS pairingsCS is a better signal or predictor of shock for the 100% CS-UCS group than for the 50% CS-UCS groupSLIDE 10: The predictive value of CS1) The two groups of rats have had an equal number of CS-UCS pairingsThus, the two groups of rats have had an equal number of CS-UCS pairings,2) CS is a better signal or predictor of shock for the 100% CS-UCS group than for the 50% CS-UCS groupBut the CS is a better signal or predictor of shock for the 100% CS-UCS group than for the 50% CS-UCS group.What did Rescorla find when he tested the two groups of rats for conditioned fear?He found that the CS elicits a much stronger response in the 100% CS-UCS group than in the 50% CS-UCS group.3) This difference must be due to the greater predictive power of the CS for the 100% groupGiven that the two groups have received an equal number of CS-UCS pairings, this difference must be due to the greater predictive power of the CS for the 100% group.4) The predictive value of a CS is an influential factor governing classical conditioningResearch conducted on signal relations have shown that the predictive value of a CS is an influential factor governing classical conditioningThis difference must be due to the greater predictive power of the CS for the 100% groupThe predictive value of a CS is an influential factor governing classical conditioning
39Response-Outcome Relations and Reinforcement Response-outcome relations and reinforcement highlight the role of cognitive processes in conditioningReinforcement is not automatic when favourable consequences follow a responseSLIDE 11: Response-Outcome Relations and Reinforcement1) Response-outcome relations and reinforcement highlight the role of cognitive processes in conditioningStudies of response-outcome relations and reinforcement highlight the role of cognitive processes in conditioningImagine that on the night before an important exam you study hard while repeatedly playing your favourite song.The next morning you get an A on your exam.Does this result strengthen your tendency to play your favourite song before exams.Probably not.You will recognize the logical relation between the response of studying hard and the reinforcement of a good grade, and only the response of studying will be strengthened.2) Reinforcement is not automatic when favorable consequences follow a response.Therefore, reinforcement is not automatic when favorable consequences follow a response.3) Individuals actively reason out the relation between responses and the outcomes that follow.We actively reason out the relation between responses and the outcomes that follow.When a response is followed by a desirable outcome, the response is more likely to be strengthened if the person thinks that the response caused the outcomeIndividuals actively reason out the relation between responses and the outcomes that followThe response is more likely to be strengthened if the person thinks that the response caused the outcome
40Response-Outcome Relations and Reinforcement (ii) Animals also engage in causal reasoningThey recognise causal relations between responses and outcomesSLIDE 12: Response-Outcome Relations and Reinforcement (ii)We might assume that only humans would engage in this causal reasoning1) Animals also engage in causal reasoning2) They recognise causal relations between responses and outcomesHowever, research suggests that under the right circumstances even pigeons can learn to recognise causal relations between responses and outcomes3) Identifying the contingencies among environmental eventsContemporary, reformulated models of conditioning view it as a matter of detecting the contingencies among environmental events.According to these theorists, organisms actively try to figure out what causes what (contingencies) in the world and around them.4) Stimuli viewed as signals that help animals minimize their aversive experiences and maximize their pleasant experiencesStimuli viewed as signals that help animals minimize their aversive experiences and maximize their pleasant experiences.The new, cognitively based theories of conditioning are quite a departure from older mainstream theories that depicted conditioning as a mindless, mechanical process.Identifying the contingencies among environmental eventsStimuli are viewed as signals that help animals minimize their aversive experiences and maximize their pleasant experiences
41Observational Learning: Basic Processes Albert Bandura (1977, 1986)Observational learningVicarious conditioning4 key processesattentionretentionreproductionmotivationacquisition vs. performanceLatent learningCognitive mapsLearned helplessnessAlbert Bandura outlined the theory of observational learning. In observational learning, vicarious conditioning occurs by an organism watching another organism (a model) be conditioned. Observational learning can occur for both classical and operant conditioning.In order for observational learning to take place, 4 key processes are at work. First the organism must pay attention to the model, retain the information observed, and be able to reproduce the behavior. Finally, an observed response is unlikely to be reproduced unless the organism is motivated to do so, i.e., they believe there will be a pay off.Bandura distinguishes between acquisition (having the response in your repertoire) and performance (actually engaging in the behavior). Bandura asserts that reinforcement usually influences already acquired responses, more than the acquisition of new responses.
42Observational Learning Mirror Neuronsfrontal lobe neurons that fire when performing certain actions or when observing another doing somay enable imitation, language learning, and empathy
431) paid 10 dollars for every 20 puzzles solved 2) studying for a class that has surprise quizzes3) ______________________ slot machines are based on this schedule4) ________________________ trolling for fish in a lake in the summer5) speed traps on highways6) _______________________ selling a product door to door7) getting the clothes out of the dryer once it buzzes8) going up a staircase to reach a landing with a nice view9) ______________________ doing 20 pushups to help stay fit
4410) _____ playing Bingo11) ______getting a paycheck at the end of 2 weeks12) _______drug testing13) ________a strike in bowling14) calling your mechanic to see if your car is fixed yet15) — frequent flyer program where one gets a free flight after a specific number of miles flown
4516) _______________________ child screams and cries in store to get what he wants—every so often it works17) ______________________ child who likes to hear theme music from Jeopardy every night at 7 pm18) trying to find a parking spot in Metropolis with a meter that works19) _______________________ wife is watching boxing match with husband- she receives a kiss at the end of every 3-minute round20) ______________________ waiting for a sunny day to go to the beach