1. Midterm 1 Oct. 6 in class Review Session after class on Monday – Location TBA

2. Read this article for Friday Oct 8th!

3. Cognitive Operations What does the brain actually do? Some possible answers: –“The mind” –Information processing… –Transforms of mental representations –Execution of mental representations of actions

4. First Principles “cognitive operations are processes that generate, elaborate upon, or manipulate representations” –As patterns of activity in one or more neurons –We often lack conscious access to these representations –Neuroscientists still know very little about how information is represented in the brain

5. Mental Representations Mental representations can start with sensory input and progress to more abstract forms –Local features such as colors, line orientation, brightness, motion are represented at low levels How might a neuron “represent” the presence of this line?

6. Mental Representations Mental representations can start with sensory input and progress to more abstract forms –Local features such as colors, line orientation, brightness, motion are represented at low levels A “labeled line” -Activity on this unit “means” that a line is present -Does the line actually have to be present?

7. Mental Representations Mental representations can start with sensory input and progress to more abstract forms –texture defined boundaries are representations arrived at by synthesizing the local texture features

8. Mental Representations Mental representations can be “embellished” - Kaniza Triangle is represented in a way that is quite different from the actual stimulus -the representation is embellished and extended

9. Mental Representations Mental Representations can be transformed –Rubin Vase, Necker Cube are examples of mental representations that are dynamic

10. Mental Representations Mental Representations can be transformed –Shepard & Metzlar (1971) mental rotation is an example of transforming a mental representation in a continuous process Mentally rotate the images to determine whether they are identical or mirror-reversed SAME MIRROR-REVERSED

11. Mental Representations Mental Representations can be transformed –Shepard & Metzlar (1971) mental rotation is an example of transforming a mental representation in a continuous process

12. Mental Representations Mental Representations can be transformed –Shepard & Metzlar (1971) mental rotation is an example of transforming a mental representation in a continuous process

13. Mental Representations Mental Representations can be transformed –Shepard & Metzlar (1971) mental rotation is an example of transforming a mental representation in a continuous process –The time it takes to respond is linearly determined by the number of degrees one has to rotate –Somehow the brain must perform a set of operations on these representations - where? how?

14. Mental Representations Mental Representations can be transformed into abstract information representations –Posner letter matching task –Are these letters from the same category (vowels or consonants) or are they different?

15. Mental Representations Mental Representations can be transformed into abstract information representations –Posner letter matching task –Are these letters from the same category (vowels or consonants) or are they different? –Are they physically the same or are they the same in an abstract way - they are in the same category? A AaAa AUAU SCSC ASAS SAME DIFFERENT

16. Mental Representations Mental Representations can be transformed into abstract information representations –Posner letter matching task –Participants are fastest when the response doesn’t require transforming the representation from a direct manifestation of the stimulus into something more abstract

17. Mental Representations Mental Representations can interfere –Stroop task: name the colour in which the word is printed (I.e. don’t read the word, just say the colour

18. Mental Representations Mental Representations can interfere –Stroop task: name the colour in which the word is printed (I.e. don’t read the word, just say the colour RED

19. Mental Representations Mental Representations can interfere –Stroop task: name the colour in which the word is printed (I.e. don’t read the word, just say the colour BLUE

20. Mental Representations Mental Representations can interfere –Stroop task: name the colour in which the word is printed (I.e. don’t read the word, just say the colour GREEN

21. Mental Representations Mental Representations can interfere –Stroop task: name the colour in which the word is printed (I.e. don’t read the word, just say the colour RED

22. Mental Representations Mental Representations can interfere –Stroop task: name the colour in which the word is printed (I.e. don’t read the word, just say the colour BLUE

23. Mental Representations Mental Representations can interfere –Stroop task: name the colour in which the word is printed (I.e. don’t read the word, just say the colour GREEN

24. Mental Representations Mental Representations can interfere –Stroop task: name the colour in which the word is printed (I.e. don’t read the word, just say the colour –The mental representation of the colour and the representation of the text are incongruent and interfere –one representation must be selected and the other suppressed –This is one conceptualization of attention

25. Mental Representations These are some examples of how a cognitive psychologist might investigate mental representations The cognitive neuroscientists asks: –where are these representations formed? –What is the neural mechanism? What is the code for a representation? –What is the neural process by which representations are transformed?

26. First Principles What are some ways that information might be represented by neurons?

27. First Principles What are some ways that information might be represented by neurons? –Magnitude might be represented by firing rate (e.g. brightness) –Presence or absence of a feature or piece of information might be represented by whether certain neurons are active or not – the “labeled line” (e.g. color, orientation, pitch) –Conjunctions of features might be represented by coordinated activity between two such labeled lines –Binding of component features might be represented by synchronization of units in a network

28. V I S I O N S C I E N C E

29. Visual Pathways Themes to notice: –Contralateral nature of visual system –Information is organized: According to spatial location According to features and kinds of information

30. Visual Pathways Image is focused on the retina Fovea is the centre of visual field –highest acuity Peripheral retina receives periphery of visual field –lower acuity –sensitive under low light

31. Visual Pathways Retina has distinct layers

32. Visual Pathways Retina has distinct layers Photoreceptors –Rods and cones respond to different wavelengths

33. Visual Pathways Retina has distinct layers Amacrine and bipolar cells perform “early” processing –converging / diverging input from receptors –lateral inhibition leads to centre/surround receptive fields - first step in shaping “tuning properties” of higher- level neurons

34. Visual Pathways Retina has distinct layers –signals converge onto ganglion cells which send action potentials to the Lateral Geniculate Nucleus (LGN) –two kinds of ganglion cells: Magnocellular and Parvocellular visual information is already being shunted through functionally distinct pathways as it is sent by ganglion cells

35. Visual Pathways visual hemifields project contralaterally –exception: bilateral representation of fovea! Optic nerve splits at optic chiasm about 90 % of fibers project to cortex via LGN about 10 % project through superior colliculus and pulvinar –but that’s still a lot of fibers! Note: this will be important when we talk about visuospatial attention

36. Visual Pathways Lateral Geniculate Nucleus maintains segregation: –of M and P cells (mango and parvo) –of left and right eyes P cells project to layers 3 - 6 M cells project to layers 1 and 2

37. Visual Pathways Primary visual cortex receives input from LGN –also known as “striate” because it appears striped when labeled with some dyes –also known as V1 –also known as Brodmann Area 17

38. Visual Pathways W. W. Norton Primary cortex maintains distinct pathways – functional segregation M and P pathways synapse in different layers

39. The Role of “Extrastriate” Areas Consider two plausible models: 1.System is hierarchical: –each area performs some elaboration on the input it is given and then passes on that elaboration as input to the next “higher” area 2.System is analytic and parallel: –different areas elaborate on different features of the input

40. The Role of “Extrastriate” Areas Different visual cortex regions contain cells with different tuning properties

41. The Role of “Extrastriate” Areas Functional imaging (PET) investigations of motion and colour selective visual cortical areas Zeki et al. Subtractive Logic –stimulus alternates between two scenes that differ only in the feature of interest (i.e. colour, motion, etc.)

42. The Role of “Extrastriate” Areas Identifying colour sensitive regions Subtract Voxel intensities during these scans… …from voxel intensities during these scans …etc. Time ->

43. The Role of “Extrastriate” Areas result –voxels are identified that are preferentially selective for colour –these tend to cluster in anterior/inferior occipital lobe

44. The Role of “Extrastriate” Areas similar logic was used to find motion-selective areas Subtract Voxel intensities during these scans… …from voxel intensities during these scans …etc. Time -> MOVING STATIONARY MOVING STATIONARY

45. The Role of “Extrastriate” Areas result –voxels are identified that are preferentially selective for motion –these tend to cluster in superior/dorsal occipital lobe near TemporoParietal Junction –Akin to Human V5

46. The Role of “Extrastriate” Areas Thus PET studies doubly-dissociate colour and motion sensitive regions

47. Electrical response (EEG) to direction reversals of moving dots generated in (or near) V5 This activity is absent when dots are isoluminant with background The Role of “Extrastriate” Areas

48. V4 and V5 are doubly-dissociated in lesion literature:

49. The Role of “Extrastriate” Areas V4 and V5 are doubly-dissociated in lesion literature: –achromatopsia (color blindness): there are many forms of color blindness cortical achromatopsia arises from lesions in the area of V4 singly dissociable from motion perception deficit - patients with V4 lesions have other visual problems, but motion perception is substantially spared

50. The Role of “Extrastriate” Areas V4 and V5 are doubly-dissociated in lesion literature: –akinetopsia (motion blindness): bilateral lesions to area V5 (extremely rare) severe impairment in judging direction and velocity of motion - especially with fast-moving stimuli visual world appeared to progress in still frames similar effects occur when M-cell layers in LGN are lesioned in monkeys

51. How does the visual system represent visual information? How does the visual system represent features of scenes? Vision is analytical - the system breaks down the scene into distinct kinds of features and represents them in functionally segregated pathways but… the spike timing matters too!

52. Visual Neuron Responses Unit recordings in LGN reveal a centre/surround receptive field many arrangements exist, but the “classical” RF has an excitatory centre and an inhibitory surround these receptive fields tend to be circular - they are not orientation specific How could the outputs of such cells be transformed into a cell with orientation specificity?

53. Visual Neuron Responses LGN cells converge on “simple” cells in V1 imparting orientation (and location) specificity

54. Visual Neuron Responses LGN cells converge on simple cells in V1 imparting orientation specificity Thus we begin to see how a simple representation - the orientation of a line in the visual scene - can be maintained in the visual system –increase in spike rate of specific neurons indicates presence of a line with a specific orientation at a specific location on the retina –Why should this matter?

55. Visual Neuron Responses Edges are important because they are the boundaries between objects and the background or objects and other objects

56. Visual Neuron Responses This conceptualization of the visual system was “static” - it did not take into account the possibility that visual cells might change their response selectivity over time –Logic went like this: if the cell is firing, its preferred line/edge must be present and… –if the preferred line/edge is present, the cell must be firing We will encounter examples in which neither of these are true! Representing boundaries must be more complicated than simple edge detection!

57. Visual Neuron Responses Boundaries between objects can be defined by color rather than brightness

58. Visual Neuron Responses Boundaries between objects can be defined by texture

59. Visual Neuron Responses Boundaries between objects can be defined by motion and depth cues