1. The ICSI/Berkeley Neural Theory of Language Project
ECG
Learning early constructions (Chang, Mok)
VERY brief overview, to situate my project:
The general tactic is to view complex cognitive phenomena as having more than one level of analysis, here 5 levels. (Cog/ling level that cogsci people study; computational level with relatively standard CS rep’ns; neural networks with structure, etc.)
Importantly, this reduction or abstraction is constrained in that structures or representations used at one level should have equivalent translations or implementations at the more concrete or biologically inspired levels. [Caveat: it’s the NTL project; “neural” is the goal, not necessarily the status quo. Any input welcome.]
e.g. SHRUTI (binding via temporal synchrony)
Most of the previous work has been at the top several levels; my research fits in there too.
I will give a brief illustration of the general approach and show how that leads naturally to the model I’ll be describing (learning early constructions).

3. Connectionist Model of Word Recognition (Rumelhart and McClelland)

4. Constraints on Connectionist Models
100 Step Rule
Human reaction times ~ 100 milliseconds
Neural signaling time ~ 1 millisecond
Simple messages between neurons
Long connections are rare
No new connections during learning
Developmentally plausible

5. Can we formalize/model these intuitions
What is a neurally plausible computational model of spreading activation that captures these features.
What does semantics mean in neurally embodied terms
What are the neural substrates of concepts that underlie verbs, nouns, spatial predicates?

6. Abstract Neuron { o u t p u t y i n p u t i w0 I0 = 1 w2 wn w1 i2 in
1 if net > 0
0 otherwise { w0
I0 = 1 w2 wn w1 i2 in i1
. . .
i n p u t i

7. Computing with Abstract Neurons
McCollough-Pitts Neurons were initially used to model
pattern classification
size = small AND shape = round AND color = green AND location = on_tree => unripe
linking classified patterns to behavior
size = large OR motion = approaching => move_away
size = small AND direction = above => move_above
McCollough-Pitts Neurons can compute logical functions.
AND, NOT, OR

8. Distributed vs Localist Rep’n
John 1
Paul
George
Ringo
John 1
Paul
George
Ringo
What are the drawbacks of each representation?

9. Distributed vs Localist Rep’n
John 1
Paul
George
Ringo
John 1
Paul
George
Ringo
What happens if you want to represent a group?
How many persons can you represent with n bits? 2^n
What happens if one neuron dies?
How many persons can you represent with n bits? n

10. Sparse Distributed Representation

11. Visual System 1000 x 1000 visual map For each location, encode:
orientation
direction of motion
speed
size
color
depth
Blows up combinatorically! …

12. Coarse Coding
info you can encode with one fine resolution unit = info you can encode with a few coarse resolution units
Now as long as we need fewer coarse units total, we’re good

13. Coarse-Fine Coding Y X G G but we can run into ghost “images”
Coarse in F2, Fine in F1
Feature 1 e.g. Orientation Y X
Y-Orientation G G
X-Orientation
but we can run into ghost “images”
Coarse in F1, Fine in F2
Y-Dir
X-Dir
Feature 2 e.g. Direction of Motion

14. Connectionist Models in Cognitive Science
Structured
PDP
Hybrid
Neural
Conceptual
Existence
Data Fitting

15. Models of Learning Hebbian ~ coincidence
Recruitment ~ one trial – Lecture 14
Supervised ~ correction (backprop)
Reinforcement ~ delayed reward
Unsupervised ~ similarity

16. Computing other relations
The 2/3 node is a useful function that activates its outputs (3) if any (2) of its 3 inputs are active
Such a node is also called a triangle node and will be useful for lots of representations.

17. Triangle nodes and McCullough-Pitts Neurons?B C A A B C

18. “They all rose” triangle nodes:
when two of the neurons fire, the third also fires
model of spreading activation

19. Spreading activation and feature structures
Parallel activation streams.
Top down and bottom up activation combine to determine the best matching structure.
Triangle nodes bind features of objects to values
Mutual inhibition and competition between structures
Mental connections are active neural connections

20. Representing concepts using triangle nodes

21. Feature Structures in Four Domains
Barrett
Ham
Container
Push
dept~CS
Color ~pink
Inside ~region
Schema ~slide
sid~001
Taste ~salty
Outside ~region
Posture ~palm
emp~GSI
Bdy. ~curve
Dir. ~ away
Schneider
Pea
Purchase
Stroll
dept~Ling
Color ~green
Buyer ~person
Schema ~walk
sid~002
Taste ~sweet
Seller ~person
Speed ~slow
emp~Gra
Cost ~money
Dir. ~ ANY
Goods ~ thing

23. 5 levels of Neural Theory of Language
Pyscholinguistic experiments
Spatial Relation
Motor Control
Metaphor
Grammar
Cognition and Language
Computation
Structured Connectionism
abstraction
Neural Net and learning
SHRUTI
Triangle Nodes
Computational Neurobiology
Biology
Neural Development
Quiz
Midterm
Finals

24. Categories and concepts- introduction
CS182/Ling109/CogSci110
Spring 2008

25. Lecture Outline Categories Aspects of a Neural Theory of concepts
Basic Level
Prototype Effects
Neural Evidence for Category Structure
Aspects of a Neural Theory of concepts
Image Schemas
Description and types
Behavioral Experiment on Image Schemas
Event Structure and Motor Schemas

26. Embodiment Alan Turing (Intelligent Machines,1948)
Of all of these fields, the learning of languages would be the most impressive, since it is the most human of these activities. This field, however, seems to depend rather too much on the sense organs and locomotion to be feasible.
Alan Turing (Intelligent Machines,1948)

27. The WCS Color Chips Basic color terms: Single word (not blue-green)
Frequently used (not mauve)
Refers primarily to colors (not lime)
Applies to any object (not blonde)

28. Concepts What Concepts Are: Basic Constraints
Concepts are the elements of reason, and
constitute the meanings of words and linguistic expressions.

29. Concepts Are:
Universal: they characterize all particular instances; e.g., the concept of grasping is the same no matter who the agent is or what the patient is or how it is done.
Stable.
Internally structured.
Compositional.
Inferential. They interact to give rise to inferences.
Relational. They may be related by hyponymy, antonymy, etc.
Meaningful.
Not tied to the specific word forms used to express them.

30. Concepts: Traditional Theory
The Traditional Theory
Reason and language are what distinguish human beings from other animals.
Concepts therefore use only human-specific brain mechanisms.
Reason is separate from perception and action, and does not make direct use of the sensory-motor system.
Concepts must be “disembodied” in this sense.

31. The neural theory
Human concepts are embodied. Many concepts make direct use of sensory-motor, emotional, and social cognition capacities of our body-brain system.
Many of these capacities are also present in non-human primates.
Continuity Principle of Am. Pragmatists

32. Classical vs prototype model of categorization
Classical model
Category membership determined on basis of essential features
Categories have clear boundaries
Category features are binary
Prototype model
Features that frequently co-occur lead to establishment of category
Categories are formed through experience with exemplars

33. Prototype theory
Certain members of a category are prototypical – or instantiate the prototype
Categories form around prototypes; new members added on basis of resemblance to prototype
No requirement that a property or set of properties be shared by all members
Features/attributes generally gradable
Category membership a matter of degree
Categories do not have clear boundaries

34. Prototype theory
Certain members of a category are prototypical – or instantiate the prototype
Category members are not all equal
a robin is a prototypical bird, but we may not want to say it is the prototype, rather it instantiates (manifests) the prototype or ideal -- it exhibits many of the features that the abstract prototype does
“It is conceivable that the prototype for dog will be unspecified for sex; yet each exemplar is necessarily either male or female.” (Taylor)

35. Prototype theory
Categories form around prototypes; new members can be added on the basis of resemblance to the prototype
Categories may also be extended on the basis of more peripheral features
house for apartment

36. Prototype theory
3. No requirement that a property or set of properties be shared by all members -- no criterial attributes
Category where a set of necessary and sufficient attributes can be found is the exception rather than the rule
Labov household dishes experiment
Necessary that cups be containers, not sufficient since many things are containers
Cups can’t be defined by material used, shape, presence of handles or function

37. Prototype theory Wittgenstein’s examination of game
Generally necessary that all games be amusing, not sufficient since many things are amusing
Board games, ball games, card games, etc. have different objectives, call on different skills and motor routines
- categories normally not definable in terms of necessary and sufficient features

38. Prototype theory
What about mathematical categories like odd or even numbers? Aren’t these sharply defined?
(Armstrong et al.) Subjects asked to assign numbers a degree of membership to the categories odd number or even number
 3 had a high degree of membership, 447 and 91 had a lower degree (all were rated at least ‘moderately good’)

39. Categories - who decides?
Embodied theory of meaning- categories are not pre-formed and waiting for us to behold them. Our need for categories drives what categories we will have
Basic level categories - not all categories have equal status. The basic level category has demonstrably greater psychological significance.

40. Basic-level categories
--Last week, Prof. Lakoff talked about prototypes, and the internal structure of individual categories and how they couldn’t just be described in terms of necessary and sufficient conditions.
--Today, I’m going to talk a bit about how categories are related to each other, particularly in terms of hierarchical structure.
-Some of the early work in this area was done by Brent Berlin, who studied categorization of plants in Tzeltal.
He found that when referring to plants, people tended to name them at the level of the genus rather than the species (e.g. oak rather than live oak) or higher level (e.g. tree).
And it was also found that Tzeltal children first learn plant names at the genus level.
Rosch and others extended this work, looking at ordinary objects in our world.
What they discovered is that not all of levels in a category hierarchy or taxonomy have equal “status”. Some levels are psychologically more basic than others.
Specifically, they found that the middle part of the hierarchy was more basic. The categories at this level have been called basic-level categories.

41. easy chair rocking chair
chair desk chair
easy chair rocking chair
furniture lamp desk lamp
floor lamp
table dining room table coffee table
Superordinate Basic Subordinate
So if you look at the hierarchical category structure for, say, furniture.
At one end, have furniture, and at the other end, very specific types of furniture {read}.
In between, in the middle level – e.g. chair, table, lamp – are basic-level categories
The other levels can be named too:
Superordinate – the level above the basic-level category, contains basic level categories
Subordinate is the level below
But in what respects are these basic-level categories considered psychologically or cognitively basic? [next slide]

42. Categories & Prototypes: Overview
Furniture
Superordinate
Sofa
Desk
Basic-Level Category
leather sofa
fabric sofa
L-shaped desk
Reception disk
Subordinate
Three ways of examining the categories we form:
relations between categories (e.g. basic-level category)
internal category structure (e.g. radial category)
instances of category members (e.g. prototypes)

43. Basic-level -- Criteria
Perception –
overall perceived shape
single mental image
fast identification
Basic in four main respects (WFDT p. 47):
The first way they are basic is in regards to perception
Perception –
The basic-level is the highest category level at which:
category membership can be determined by overall perceived shape.
A single mental image can represent the entire category,
 applies to chair, but not to furniture
So shape plays an important role…
People can identify category members fastest at the basic level

44. Basic-level -- Criteria
Perception
Function – motor program for interaction
Another respect these are basic is in terms of functional interaction.
e.g. sit on chair – changing posture, bending legs
put things on tables, move things around on them
turn lights on and off
--No set motor routine for interacting with furniture,
--And same motor routine for different subordinate types

45. Basic-level -- Criteria
Perception
Function
Words –
shortest
first learned by children
first to enter lexicon
A third respect is in terms of communication.
The words used to label these categories are READ

46. Basic-level -- Criteria
Perception
Function
Communication
Knowledge organization –
most attributes are stored at this level
A fourth respect is in terms of how knowledge is organized or stored.
Superordinate level -- Very few attributes stored – furniture – usually bigger than a breadbox? Used inside?
Basic level
- most attributes –know a lot about things at this level -- Chairs have seats, legs, you sit on them, etc…
Subordinate – few additional attributes (unless an expert)
Can also think in terms of similarities and differences
-Many more differences between basic-level categories than subordinate categories, e.g. chairs and lamps, vs. desk chairs and an easy chairs.
-And there is more similarity within basic-level categories than within superordinate categories e.g.
And other categories can be analysed this way -- CLASS SUGGESTIONS? NAME AN OBJECT

47. Basic-Level Category Perception: Function:
What constitutes a basic-level category?
Perception:
similar overall perceived shape
single mental image
(gestalt perception)
fast identification
Function:
general motor program
Communication:
shortest
most commonly used
contextually neutral
first to be learned by children
first to enter the lexicon
Knowledge Organization:
most attributes of category members stored at this level

48. Other Basic-level categories
Objects
Colors
Motor-routines
The criteria just listed seem to apply mostly to objects. But the notion of basic-level can be applied to other domains as well
Basic color terms met many of these same criteria in terms of short words (red vs. vermillion), earliest learned, and fast identification.
And similar things might be done with actions, e.g. basic motor actions might be ??? [walk, run – not saunter and sprint]

49. Concepts are not categorical

50. Mother The birth model The genetic model The nurturance model
The person who gives birth is the mother
The genetic model
The female who contributes the genetic material is the mother
The nurturance model
The female adult who nurtures and raises a child is the mother of the child
The marital model
The wife of the father is the mother
The genealogical model
The closest female ancestor is the mother
(WFDT Ch.4, p.74, p.83)

51. Radial Structure of Mother
Genetic mother
Stepmother
Unwed mother
Adoptive mother
Central Case
Surrogate mother
Birth mother
Biological mother
Natural mother
Foster mother
The radial structure of this category is defined with respect to the different models

52. Marriage What is a marriage?
What are the frames (or models) that go into defining a marriage?
What are prototypes of marriage?
What metaphors do we use to talk about marriages?
Why is this a contested concept right now?

53. Concepts and radial categories
Concepts can get to be the "prototype" of their category in various ways.
Central subcategory (others relate to this)
Amble and swagger relate to WALK
Shove relates to PUSH
Essential (meets a folk definition: birds have feathers, beaks, lay eggs)
Move involves change of location.
Typical case (most are like this: "sparrow")
Going to a conference involves air travel.
Ideal/anti-ideal case (positive social standard: "parent"); anti-ideal case (negative social standard: "terrorist")
Stereotype (set of attributes assumed in a culture: "Arab")
Salient exemplar (individual chosen as example)

54. Category Structure Classical Category: Radial Category:
necessary and sufficient conditions
Radial Category:
a central member branching out to less-central and non-central cases
degrees of membership, with extendable boundary
Family Resemblance:
every family member looks like some other family member(s)
there is no one property common across all members (e.g. polysemy)
Prototype-Based Category
Essentially-Contested Category (Gallie, 1956) (e.g. democracy)
Ad-hoc Category (e.g. things you can fit inside a shopping bag)
My original example for an ad-hoc category was “things you’ll bring to a picnic”, but a keen student pointed out that it is in fact a frame-based category (the picnic frame).

55. Prototype Cognitive reference point Social stereotypes
standards of comparison
Social stereotypes
snap judgments
defines cultural expectations
challengeable
Typical case prototypes
default expectation
often used unconsciously in reasoning
Ideal case / Nightmare case
e.g. ideal vacation
can be abstract
may be neither typical nor stereotypical
Paragons / Anti-paragons
an individual member that exhibits the ideal
Salient examples
e.g. 9/11 – terrorism act
Generators
central member + rules
e.g. natural number = single-digit numbers + arithmetic

56. Neural Evidence for category structure
Are there specific regions in the brain to recognize/reason with specific categories?

57. Category Naming and Deficits
People with brain injury have selective deficits in their knowledge of categories.
Some patients are unable to identify or name man made objects and others may not be able to identify or name natural kinds (like animals)

58. A PET Study on categories (Nature 1996)

59. Study
16 adults (8M, 8F) participated in a PET (positron emission tomography) study.
Involves injecting subject with a positron emitting radioactive substance (dye)
Regions with more metabolic activity will absorb more of the substance and thus emit more positrons
Positron-electron collisions yield gamma rays, which are detected
Increased rCBF (regional changes in cerebral blood flow) was measured
When subjects viewed line drawings of animals and tools.

60. The experiment
Subjects looked at pictures of animals and tools and named them silently.
They also looked at noise patterns (baseline 1)
And novel nonsense objects (baseline 2)
Each stimulus was presented for 180ms followed by a fixation cross of 1820 ms.
Drawings were controlled for name frequency and category typicality

61. medial
lateral

64. Left middle temporal gyrus
ACC
Premotor

65. Calcarine Sulcus

66. Conclusions
Both animal and tool naming activate the ventral temporal lobe region.
Tools differentially activate the ACC, pre-motor and left middle temporal region (known to be related to processing action words).
Naming animals differentially activated left medial occipital lobe (early visual processing)
The object categories appear to be in a distributed circuit that involves activating different salient aspects of the category.