Adaptive behavior in autonomous individuals Michael Vigorito Department of Psychology

History of Mind Phrenology – Franz Gall (1758 – 1828) suggested that bumps of the skull represented mental abilities. His theory, though incorrect, nevertheless proposed that different mental abilities were modular. Studying the mind (intelligence) from the perspective of information processing (Cognitive Psychology). Studying intelligence from the perspective of behavioral adaptation to the environment

Neural Communication Neuroscientists and psychologists understand that humans and animals operate similarly when responding to environmental stimuli and when processing information with some exceptions (e.g., language processing) Today we know that much of animal/human behavior is a result of numerous complex circuits that function in parallel.

A simple nervous system circuit 1.Sensory neuron 2.Motor neuron 3.Interneuron Behavior as an adaptation to the environment Adaptive Behavior does not always require information processing (cognition)

Rat in an operant Chamber Anthropomorphism is a common error when explaining animal behavior We sometimes overestimate the importance of our cognition (thinking) in the control of our own behavior. Behavior change (Learning) as an adaptation to environmental change.

Eliciting Stimulus Elicited Response Unconditioned (innate) Behaviors Simple => (e.g., reflexes) Heat Limb withdrawal Air puff to eye Eye blink Food in mouth Salivation Loud noise Startle Complex => [e.g., sign stimuli & FAP ] Taste of Food Chewing/swallowing (consummatory behavior) Smell of food Approach and search (appetitive behavior) Long beak with red spot Pecking response Visual stimulus Eye fixation Egg Egg retrieval Very Complex => e.g. Courtship and reproductive behavior in the stickleback e.g. General Search Focal Search Food Handling/Ingestion Behavior Systems Ethology

Valentino Braitenberg's booklet 'Vehicles' has received much attention in the robotics community. Shows that complex behavior does not require a complex brain, but emerges from the interactions between the vehicle and its environment. The more complex its environment, the more complex its behavior. Law of Uphill Analysis and Downhill Invention: machines are easy to understand if you’re creating them; much harder to understand "from the outside". Psychological consequence: if we don’t know the internal structure of a machine, we tend to overestimate its complexity. Valentino Braitenberg’s Vehicles

Components: - One heat sensor - One motor (rotates in forward direction only) - One connection between sensor & motor Principle: The more there is of the quality (e.g., heat) to which the sensor is tuned, the faster the motor goes. Description: alive, restless, doesn’t "like" heat Vehicle 1: Alive

Components: 2 light sensors, 2 motors, each sensor connected to the motor on the same side ("uncrossed") Principle: The more there is of the quality to which the sensor is tuned, the faster the motors go ("excitatory"). Vehicle 2a: Cowardly ++

Description: dislikes source to which the sensor is tuned; occasionally "attacks" it Vehicle 2a: Cowardly

Components: 2 sensors, 2 motors, each sensor connected to the motor on the opposite side ("crossed") Principle: The more there is of the quality to which the sensor is tuned, the faster the motors go ("excitatory"). Description: dislikes source to which the sensor is tuned; "attacks" it Vehicle 2b: Aggressive ++

+ + Vehicle 2a: Coward + +

- - Vehicles 3a & 3b Loving Vehicle 3a : Loving/Quietly accepting Components: uncrossed connections Description: loves the source, wants to be near it, comes to rest facing it Vehicle 3b: Loving/Exploring Components: crossed connections Description: likes the source, but easily attracted away -- Principle: The more there is of the quality to which the sensor is tuned, the slower the motors go ("inhibitory").

- - Vehicles 3a & 3b Loving Vehicle 3a : Loving/Quietly accepting Components: uncrossed connections Description: loves the source, wants to be near it, comes to rest facing it Vehicle 3b: Loving/Exploring Components: crossed connections Description: likes the source, but easily attracted away -- Principle: The more there is of the quality to which the sensor is tuned, the slower the motors go ("inhibitory").

VV U U Components: 4 sensors, 4 motors, each tuned to different properties of the environment. Principle: one each of the four types so far: uncrossed/excitatory: tuned to temperature crossed/excitatory: tuned to light uncrossed/inhibitory: tuned to organic material crossed/inhibitory: tuned to oxygen level Description: Cowardly toward areas of high temperature; Aggressive toward light sources; Loves organic material; leaves and seeks new source if environment is depleted; Restlessly seeks best source of oxygen. Vehicle 3c appears to know a great deal and appears to have a system of Values Vehicle 3c: Knowing, Valuing Excitatory Inhibitory

Vehicle 4a: Displaying Instincts, Specialization Motor Speed Intensity of Stimulation Excitatory Inhibitory Motor Speed Intensity of Stimulation Motor Speed Intensity of Stimulation Excitatory Inhibitory Monotonic relationship Non-Monotonic relationship

Components: sensors and motors Principle: connections both excitatory and inhibitory but non-monotonic Description: does everything 3c vehicles do, but with much less predictability. Vehicle 4a: Displaying Instincts, Specialization

Vehicle 4b: Making Decisions Components: sensors and motors, and threshold devices Principle: connections both excitatory and inhibitory but non- monotonic Description: does everything 3c vehicles do, but with much less predictability, and appears to ponder over its "decisions"; appears to will. Intensity of Stimulation Motor Speed

Vehicle 5: Reasoning Logically, Counting, Recognizing Individuals Components: sensors, motors, threshold devices, some networked, so they give output according to some numerical formula (e.g., one out for every three in). Principle: connections are both excitatory and inhibitory, monotonic and non-monotonic. Like a computer. Description: apparently recognizes individuals, counts, does logic, math calculations.

Learning Change in Behavior as a result of experience Existing reflexes, FAP, and complex behavioral systems are modified as a result of experience Learning = behavioral adaptation to changing environments –mobility results in a changing environment –the presence of autonomous others results in a changing environment

Eliciting Stimulus Elicited Response Emitted Response Environmental Stimuli Internal Stimuli motor feedback motivation (extrinsic; intrinsic) emotion cognition (e.g. expectation; intention)

*Eliciting Stimulus Elicited Response Emitted Response *Environmental Stimuli Types of learning (& Laboratory Procedures for investigating them): 1.Habituation/Sensitization 2.Classical (Pavlovian) conditioning 3.Operant (Instrumental) conditioning 4.Cognitive Learning

*Eliciting Stimulus Elicited Response *Environmental Stimuli Emitted Response Not Required Habituation In an habituation experiment the eliciting stimulus is presented more than once or repeatedly.

Habituation Gill Withdrawal Reflex Habituation in a rat

Sensitization

Role of Habituation & Novelty in Exploratory Behavior Given a choice between an object that was previously encountered and a second novel item, rats and monkeys prefer the novel item. This increases exploration of objects in the environment Part 1: exposure to single object Part 2: exposure to both objects Rat spends more time exploring novel object

*Eliciting Stimulus Elicited Response *Environmental Stimuli Emitted Response Not Required Classical Conditioning – associating two stimuli

*Eliciting Stimulus Elicited Response *Environmental Stimuli Emitted Response Not Required *Neutral Stimulus (Light) + FoodSalivationRestrained dog Food Approach & consumption Mobile pigeon Classical Conditioning – associating two stimuli

Sign Tracking revealed through a classical conditioning procedure Sign Tracking as Adaptive Behavior Sign tracking Video The form of the pecking behavior Video Sign Tracking as Maladaptive Behavior Sign tracking long box experiment Video

Socially Learned Food Preferences in rodents Novel Food A Part 1: Demonstrator rat eats a novel food alone Observer ratDemonstrator Part 2: Observer rat socially interacts with demonstrator after the demonstrator is finished eating Novel Food BNovel Food A vs. Part 3: Observer rat prefers the food eaten by the demonstrator rat How do they learn? Lingering smell of novel food is associated with Carbon Disulfide smell on Demonstrator’s breadth (Stimulus 1)(Stimulus 2)

Operant Conditioning –Associate responses and reinforcement (eliciting stimulus) –Organism is active (responses are required) –Responses that emerge or are strengthened are said to be emitted

*Eliciting Stimulus Elicited Response Emitted Response *Environmental Stimuli Eliciting Stimulus if effective is a reinforcer. The emitted response (Operant) is increased in probability when followed by a reinforcer.

Operant Conditioning Shaping Video Skinner explains operant conditioning

Copyright © 2005 Allyn & Bacon Cognitive Learning (Learning in the absence of eliciting stimuli) Latent learning –Tolman’s rats: cognitive maps Learning Information that may (or may not) be useful in the future

Spatial Learning in the Water maze Task First exposure to the water maze After several trials