1. Neural representation and decoding of the meanings of words (continued)

2. Some PubMed search tips that you might not already know
Selecting by subfield, e.g. title
Boolean searches and wildcards
Different ways of referring to same thing
Review tag

3. Ways of representing meaning
Corpus semantics / distributional semantics
See how words co-occur with each other in large bodies of text
Semantic network
Relations between words: “is-a”, etc.
WordNet
Embodied semantics
Meanings are grounded in sense modalities

4. The Mitchell study: word stimuli and semantic features
Stimuli: concrete nouns
E.g. hammer, shirt, dog, celery (60 words in all)
12 categories (tools, clothing, food, etc.), each with 5 words
Semantic features: action verbs
E.g. push, move, taste, see (25 semantic features in all)
Each noun has a 25-element semantic feature vector of its co-occurrence freqs with the verbs, from Google text- corpus
hammer = 0.13*break *touch *eat + …
celery = *break *touch *eat + …

5. Example co-occurrence features
Features for cat:
say said says (0.592),
see sees (0.449),
eat ate eats (0.435),
run ran runs (0.303),
hear hears heard (0.208),
open opens opened (0.175),
smell smells smelled (0.163),
clean cleaned cleans (0.146),
move moved moves (0.088),
listen listens listened (0.075),
touch touched touches (0.075) …

6. Mitchell model architecture: represent nouns in terms of semantic features (action verbs)
fMRI data and semantic features publicly available at

7. Interpolating between stimuli, using a model of the stimulus space
Pattern-information analysis: from stimulus decoding to computational-model testing. Kriegeskorte N. Neuroimage May 15;56(2):

8. Corpus based approach: pros and cons
Advantages:
Works quite well in practice
Used a lot in computer language processing
Good for capturing semantic relations between single words
Disadvantages:
Unclear how to relate it to neural representations
Unclear how to handle logical relations between words

9. Semantic network

10. WordNet Founded by George Miller (“magic number 7”)

11. Huth et al. (2012) Semantic space in cortex

12. Representing categories in WordNet

13. Principal Components Analysis (PCA)

14. Representing multiple semantic principal components

15. A closer look at semantic space

16. What do the components mean?

17. Highly distributed representations

18. How much of each region’s activation does the model explain?

19. Embodied theory of meaning
Words are represented in terms of bodily sense modalities: vision, hearing, movement, etc.
Barsalou, L. W. (2008). Grounded cognition. Annu. Rev. Psychol., 59,
Pulvermüller, F. (2013). How neurons make meaning: brain mechanisms for embodied and abstract-symbolic semantics. Trends in cognitive sciences, 17(9),
Binder, J. R., & Desai, R. H. (2011). The neurobiology of semantic memory. Trends in cognitive sciences, 15(11),

20. Embodied theory of meaning
Binder, J. R., & Desai, R. H. (2011). The neurobiology of semantic memory. Trends in cognitive sciences, 15(11),

21. Embodied theory of meaning
Pulvermüller, F. (2013). How neurons make meaning: brain mechanisms for embodied and abstract-symbolic semantics. Trends in cognitive sciences, 17(9),

22. Example: somatotopic representation of motor words

23. Example: somatotopic representation of motor words
Pulvermüller, F., Trends in Cog Sci (2013).

24. Embodied-looking activation shows up even using corpus statistics
“Gustatory cortex” for celery in Mitchell et al. (2008)
Mouth / toungue areas

25. What about abstract words?
Pulvermüller, F., Trends in Cog Sci (2013).

26. Higher level “abstraction” areas?
Pulvermüller, F., Trends in Cog Sci (2013).

27. Lots of open questions! Composition of meaning:
How does the brain build a representation of “The child threw the ball” out of its representations of “child”, “threw” and “ball”?
Systematicity / compositionality
The brain can recombine words into a potentially unlimited number of new sentences. How?
Syntax
How does the brain represent “the cat chased the dog” vs “the dog chased the cat” ?