Download presentation
Presentation is loading. Please wait.
1
Emergence of Semantics from Experience
Jay McClelland Department of Psychology and Center for Mind, Brain, and Computation Stanford University 1
2
The Parallel Distributed Processing Approach to Semantic Cognition
Representation is a pattern of activation distributed over neurons within and across brain areas. Bidirectional propagation of activation underlies the ability to bring these representations to mind from given inputs. The knowledge underlying propagation of activation is in the connections. Experience affects our knowledge representations through a gradual connection adjustment process language 2
3
Distributed Representations: and Overlapping Patterns for Related Concepts
dog goat hammer dog goat hammer 3
4
Emergence of Meaning in Learned Distributed Representations
Learned distributed representations that capture important aspects of meaning emerge through a gradual learning process in simple connectionist networks The progression of learning captures several aspects of cognitive development: Differentiation of Concepts Illusory Correlations Overgeneralization And many other things 4
5
5
6
The Rumelhart Model 6
7
The Training Data: All propositions true of items at the bottom level of the tree, e.g.: Robin can {grow, move, fly} 7
8
Target output for ‘robin can’ input
8
9
Forward Propagation of Activation
aj ai wij neti=Sajwij wki 9
10
Back Propagation of Error (d)
aj wij ai di ~ Sdkwki wki dk ~ (tk-ak) Error-correcting learning: At the output layer: Dwki = edkai At the prior layer: Dwij = edjaj … 10
11
11
12
12
13
Early Later Later Still E x p e r i e n c e 13
14
Why Does the Model Show Progressive Differentiation?
Learning is sensitive to patterns of coherent covariation Coherent Covariation: The tendency for properties of objects to co-vary in clusters Figure shows attribute loadings on the principal dimensions of covariation. These capture: 1. Plants vs. animals 2. Birds vs. fish 3. Trees vs. flowers Same color = features that covary Diff color = anti-covarying features 14
15
Trajectories of Concept Representations During Differentiation
15
16
Illusory Correlations
16
17
A typical property that a particular object lacks
e.g., pine has leaves An infrequent, atypical property 17
18
Overgeneralization of Frequent Names to Similar Objects
“tree” “goat” “dog” 18
19
19
20
Other Applications of the Model
Expertise effects Conceptual reorganization Effects of language and culture Effects of brain damage: Loss of differentiation Overgeneralization in object naming Illusory correlations camel swan 20
21
Conclusion We represent objects using patterns of activity over neuron-like processing units These patterns depend on connection weights learned through experience Differences in experience lead to differences in conceptual representations. 21
Similar presentations
