1. CSE 153 Cognitive ModelingChapter 3 Representations and Network computations In this chapter, we cover: –A bit about cortical architecture –Possible representational strategies the brain might use –How networks of neurons can do interesting things.

2. CSE 153 Cognitive ModelingChapter 3 Representations and Network computations In this chapter, we cover: –A bit about cortical architecture –Possible representational strategies the brain might use –How networks of neurons can do interesting things.

3. CSE 153 Cognitive ModelingChapter 3 The cortex The cortex is a sheet scrunched up to fit inside your head Cortex (cortical sheet) White matter (connections)

4. CSE 153 Cognitive ModelingChapter 3 The cortex is layered There are about six layers These can roughly be thought of as “input” “hidden” and “output”.

5. CSE 153 Cognitive ModelingChapter 3 Cortical architecture The cortex is also organized into cortical columns (Vernon Mountcastle) –If you move laterally across the surface of the brain, the neurons respond to different stimuli, often in a smoothly varying way –If you move vertically into the layered cortex, all of the neurons appear to respond to exactly the same stimulus. For example in the early visual system, there are orientation columns (where neurons respond to bars of a particular orientation) Interleaved with ocular dominance columns

6. CSE 153 Cognitive ModelingChapter 3

7. CSE 153 Cognitive ModelingChapter 3 Ocular dominance columns

8. CSE 153 Cognitive ModelingChapter 3 The cortex is layered

9. CSE 153 Cognitive ModelingChapter 3 The cortex is layered The thickness of the layers varies from area to area of the brain. These can be clues to their function! A is from primary visual - “input” layer (4) is emphasized B&C are from later visual cortex - “hidden” layers 2-3 emphasized D is from motor cortex - “output” layers 5-6 emphasized.

10. CSE 153 Cognitive ModelingChapter 3 Summary interpretation of one column Note just three areas in one column. Highly oversimplified view - but justifies choices made later in the models… This is not the whole story - how do these connect to other columns?

11. CSE 153 Cognitive ModelingChapter 3 Communication between columns/areas Feedforward is from hidden in one area to input in “later” or “downstream” area. Feedback is from hidden->hidden & output Lateral connections from hidden and output to everything.

12. CSE 153 Cognitive ModelingChapter 3 Representations and Network computations In this chapter, we cover: –A bit about cortical architecture –Possible representational strategies the brain might use –How networks of neurons can do interesting things.

13. CSE 153 Cognitive ModelingChapter 3 How are things represented? Is “B” just the last neuron in this chain?

14. CSE 153 Cognitive ModelingChapter 3 How are things represented? Q: Which neuron represents this face? A: All of them! This is called a distributed representation

15. CSE 153 Cognitive ModelingChapter 3 Distributed representations This has two components: Every neuron represents more than one thing Every thing is represented by more than one neuron. One view is that the way each neuron represents more than one thing is based on how similar the input is to its pattern (what it is a detector for).

16. CSE 153 Cognitive ModelingChapter 3 Graded responses in Macaque These are spike histograms from a single neuron in response to the stimuli above them. (Keiji Tanaka, 1996) This is an interpretation of what this neuron is looking for - four areas of curvature.

17. CSE 153 Cognitive ModelingChapter 3 Distributed responses in Macaque These are maps of inferotemporal cortex showing areas where neurons have been found to be responsive to each of these 8 stimuli (Keiji Tanaka, 2003) Note they are distributed across cortex!

18. CSE 153 Cognitive ModelingChapter 3 Distributed responses in Human This is from a now-classic paper by Jim Haxby and colleagues showing patterns of activation to different categories of stimuli. These are differences from the mean activation. Note they are distributed across cortex!

19. CSE 153 Cognitive ModelingChapter 3 One way to form a distributed representation: Coarse Coding Each neuron codes for a range of values It fires in proportion to the similarity of the input to its pattern. Neurons overlap in what they represent Hence these three neurons can represent a range of colors!

20. CSE 153 Cognitive ModelingChapter 3 The other kind of representation: Localist Each neuron codes for a single thing It fires in proportion to the similarity of the input to its pattern, or P(my-pattern|input) How is this different from distributed? It is a matter of what you consider your level of analysis! The face processing network shown earlier has a localist representation of names, but the name units are part of a distributed representation of the face!

21. CSE 153 Cognitive ModelingChapter 3 Face processing simulation

22. CSE 153 Cognitive ModelingChapter 3 Face processing simulation points Neurons can re-represent the input in ways necessary to solve the task This was a very simple simulation to make the re- representation obvious The same point was made in the expertise network - the neurons re-represented the category of expertise by spreading it out, and re-represented other categories by collapsing them. Localist representations are useful fictions to use as outputs - so we can tell when the network is right!

23. CSE 153 Cognitive ModelingChapter 3 Face processing simulation points Another way of thinking about re-representation is that the networks: –Emphasize distinctions: For expressions, faces of the same person are separated, even though they have perceptual overlap –Collapse distinctions: For expressions, faces of different people are grouped - even though they “look different.” –Vice-versa for identity: identities are distinguished, expressions are collapsed.

24. CSE 153 Cognitive ModelingChapter 3 Representations and Network computations In this chapter, we cover: –A bit about cortical architecture –Possible representational strategies the brain might use –How networks of neurons can do interesting things.

25. CSE 153 Cognitive ModelingChapter 3 Network computations What can we do with networks? –Categorize Last week, we briefly discussed the face network simulation –Bi-directional activation dynamics Earlier today, we discussed attractor networks. These are useful as: –Pattern completers –Ambiguity resolvers (which meaning did I mean?) –Memory models They work via a process of constraint satisfaction –Inhibitory competition Helps implement constraint satisfaction Helps develop sparse, distributed representations during learning

26. CSE 153 Cognitive ModelingChapter 3 Network computations What can we do with networks? –Categorize Last week, we briefly discussed the face network simulation: –This reminds me that I wanted to talk more about that - so, briefly I want to go over what some of the analysis means.

27. CSE 153 Cognitive ModelingChapter 3 Network computations What can we do with networks? –Inhibitory competition

28. CSE 153 Cognitive ModelingChapter 3 Types of Inhibition Feedforward and feedback Reacts to excitationAnticipates excitation

29. CSE 153 Cognitive ModelingChapter 3 Types of Inhibition Feedforward and feedback Inhibition simulation

30. CSE 153 Cognitive ModelingChapter 3 The k-WTA Approximation Computationally expensive to simulate inhibitory interneurons: –Extra units and connections. –Slower rate constants (dt) required to avoid oscillations (many cycles of updating to process inputs). Thus, we approximate inhibition by only activating k units. Approximates set point behavior: thermostat setting. Compute g i such that k units are above threshold, the rest below.

31. CSE 153 Cognitive ModelingChapter 3 k-WTA approximation 1.Rank order the units by activation 2.Compute the level of g i that is necessary to turn off the bottom N-k units. 3.Inject that into the units. The above is illustrating how this affects different distributions of activation.

32. CSE 153 Cognitive ModelingChapter 3 Average k-WTA approximation 1.Rank order the units by activation 2.Compute the average activation of the bottom N-k and top k 3.Set the g i to a value a fraction of the way between these (set by “pt” above) The above is illustrating how this approach allows a flexible number of different distributions of activation.

33. CSE 153 Cognitive ModelingChapter 3 Questions?