1. Development and Disintegration of Conceptual Knowledge: A Parallel-Distributed Processing Approach
James L. McClelland
Department of Psychology and Center for Mind, Brain, and Computation Stanford University

2. Parallel Distributed Processing Approach to Semantic Cognition
Representation is a pattern of activation distributed over neurons within and across brain areas.
Bidirectional propagation of activation mediated by a learned internal representation underlies the ability to bring these representations to mind from given inputs.
The knowledge underlying propagation of activation is in the connections, and is acquired through a gradual learning process.
language

3. A Principle of Learning and Representation
Learning and representation are sensitive to coherent covariation of properties across experiences.

4. What is Coherent Covariation?
The tendency of properties of objects to co-occur in clusters.
e.g.
Has wings
Can fly
Is light Or
Has roots
Has rigid cell walls
Can grow tall

5. Development and Degeneration
Sensitivity to coherent covariation in an appropriately structured Parallel Distributed Processing system creates the taxonomy of categories that populate our minds and underlies the development of conceptual knowledge.
Gradual degradation of the representations constructed through this developmental process underlies the pattern of semantic disintegration seen in semantic dementia.

6. Some Phenomena in Development
Progressive differentiation of concepts
Overextension of frequent names
Overgeneralization of typical properties

8. The Rumelhart Model

9. The Training Data:
All propositions true of items at the bottom level of the tree, e.g.:
Robin can {grow, move, fly}

10. Target output for ‘robin can’ input

11. Forward Propagation of Activationaj ai
wij
neti=Sajwij
wki

12. Back Propagation of Error (d)aj
wij ai
di ~ Sdkwki
wki
dk ~ (tk-ak)
Error-correcting learning:
At the output layer: Dwki = edkai
At the prior layer: Dwij = edjaj …

15. Early
Later
Later Still
E x p e r i e
n c e

17. What Drives Progressive Differentiation?
Waves of differentiation reflect coherent covariation of properties across items.
Patterns of coherent covariation are reflected in the principal components of the property covariance matrix.
Figure shows attribute loadings on the first three principal components:
1. Plants vs. animals
2. Birds vs. fish
3. Trees vs. flowers
Same color = features covary in component
Diff color = anti-covarying features

18. Coherence Training Patterns Items Properties Coherent Incoherent
is can has is can has …
Items
No labels are provided
Each item and each property occurs with equal frequency
Coherently co-varying inputs are not presented at the same time!

19. Effect of Coherence on Representation

20. Overextension of A Frequent Name to Similar Objects
Oak
Goat
“tree”
“goat”
“dog”

22. Overgeneralization of typical properties
Rochel Gelman found that children think that all animals have feet.
Even animals that look like small furry balls and don’t seem to have any feet at all.

23. A typical property that a particular object lacks
e.g., pine has leaves
An infrequent,
atypical property

24. Development and Degeneration
Sensitivity to coherent covariation in an appropriately structured Parallel Distributed Processing system underlies the development of conceptual knowledge.
Gradual degradation of the representations constructed through this developmental process underlies the pattern of disintegration seen in semantic dementia.

25. Disintegration of Conceptual Knowledge in Semantic Dementia
Progressive loss of specific knowledge of concepts, including their names, with preservation of general information
Overextension of frequent names
Overgeneralization of typical properties

26. Picture naming and drawing in
Sem. Demantia

28. Grounding the Model in What we Know About The Organization of Semantic Knowledge in The Brain
Specialized areas for each of many different kinds of semantic information.
Semantic dementia results from progressive bilateral disintegration of the anterior temporal cortex.
Destruction of the medial temporal lobes results in loss of memory for recent events and loss of the ability to form new memories quickly, but leaves existing semantic knowledge unaffected.
language

29. Proposed Architecture for the Organization of Semantic Memory
action
name
Medial Temporal Lobe
motion
Temporal
pole
color
valance
form

30. Rogers et al (2005) model of semantic dementia
Trained with 48 items from six categories (from a clinical test).
Names are individual units, other patterns are feature vectors.
Features come from a norming study.
From any input, produce all other patterns as output.
Representations undergo progressive differentiation as learning progresses.
Test of ‘picture naming’: Present vision input.
Most active name unit above a threshold is chosen as response.
name
assoc
function
temporal pole
vision

31. Errors in Naming for As a Function of Severity
Simulation Results
Patient Data
omissions
within categ.
superord.
Severity of Dementia
Fraction of Connections Destroyed

32. Simulation of Delayed Copying
Visual input is presented, then removed.
After three time steps, the vision layer pattern is compared to the pattern that was presented.
Omissions and intrusions are scored for typicality.
name
assoc
function
temporal pole
vision

33. Omission Errors
IF’s ‘camel’

34. Intrusion Errors
DC’s ‘swan’

35. Development and Degeneration
Sensitivity to coherent covariation in an appropriately structured Parallel Distributed Processing system underlies the development of conceptual knowledge.
Gradual degradation of the representations constructed through this developmental process underlies the pattern of semantic disintegration seen in semantic dementia.

37. A Hierarchical Bayesian Characterization
Initially, assume there’s only one ‘kind of thing’ in the world.
Assign probabilities to occurrence of properties according to overall occurrence rates.
Coherent covariation licenses the successive splitting of categories
Probabilities of individual features become much more predictable, and conditional relations between features are available for inference.
Overgeneralization and overextension are consequences of implicitly applying the ‘kind’ feature probabilities to the ‘item’ and depends on the current level of splitting into kinds.
This process occurs
By an on-line, incremental learning process.
In a gradual and graded way.
Without explicit enumeration of possibilities.

38. A Hierarchical Bayesian Characterization (Cont’d)
Effect of damage causes the network to revert to a simpler model.
Perhaps the network can be seen as maximizing its accuracy in ‘explaining’ properties of objects given
Limited training data during acquisition
Limited resources during degredation

39. Thanks for your attention!

40. Sensitivity to Coherence Requires ConvergenceA A A