1. Brain Mechanisms of Unconscious Inference J. McClelland Symsys 100 April 22, 2010

2. Last time… We considered many examples of unconscious inference –Size illusions –Illusory contours –Perception of objects from vague cues –Unconsious associative priming –Lexical effects on speech perception –Effect of visual speech on speech perception.

3. Today… We ask about the mechanisms through which this occurs

4. Three Problems for Unconscious Inference Theory GARY HATFIELD, Philosopher, Univ. of Penna. The cognitive machinery problem: Are the unconscious inferences posited to explain size perception and other phenomena carried out by the same cognitive mechanisms that account for conscious and deliberate inferences, or does the visual system have its own inferential machinery? In either case, what is the structure of the posited mechanisms? The sophisticated content problem: how shall we describe the content of the premises and conclusions? For instance, in size perception it might be that the premises include values for visual angle and perceived distance […]. But shall we literally attribute concepts of visual angle […] to the visual system? The phenomenal experience problem: Third, to be fully explanatory, unconscious inference theories of perception must explain how the conclusion of an inference about size and distance leads to the experience of an object as having a certain size and being at a certain distance. In other words, the theories need to explain how the conclusion to an inference […] can be or can cause perceptual experience.

5. Proposed answers to these questions The cognitive machinery problem. The machinery of unconscious inference is the propagation of activation among neurons. Neurons embedded in the perceptual system can carry out such inferences without engaging the mechanisms used in conscious and deliberative inference. The sophisticated content problem. Activation of particular neurons or groups of neurons codes for particular content. Connections among neurons code the conditional relationships between items of content. The phenomenal experience problem. Activity of certain populations of neurons is a necessary condition for conscious experience. Anything that affects the activation of they neurons will affect conscious experience. Is this activity the actual substrate of experience itself?

6. Outline of Lecture Neurons: Structure and Physiology Neurons and The Content of Experience How Neurons Make Inferences –And how these capture features of Bayes Rule Integration of Information in Neurons and in Perception

7. Neuronal Structure and Function Neurons combine excitatory and inhibitory signals obtained from other neurons. They signal to other neurons primarily via ‘spikes’ or action potentials.

8. Neurons and the Content of Experience The doctrine of specific nerve energies: –Activity of specific neurons corresponds to specific sensory experiences Touch at a certain point on the skin Light at a certain point in the visual field The brain contains many ‘maps’ in which neurons correspond to specific points On the skin In the visual world In non-spatial dimensions such as auditory frequency If one stimulates these neurons in a conscious individual, an appropriate sensation is aroused. If these neurons are destroyed, a corresponding void in experience occurs. –Visual ‘scotomas’ arise from lesions to the maps in primary visual cortex.

9. Feature Detectors in Visual Cortex Line and edge detectors in primary visual cortex (classic figure at left). –Cells show a graded response depending on exact orientation of line. Representation of motion in area MT –Destroy MT on one side of the brain, and perception of motion in the opposite side of space is greatly impaired.

10. Neural Representations of Objects and their Identify The ‘Grandmother Cell’ hypothesis: –Is there a dedicated neuron, or set of neurons, for each cognized object, such as my Grandmother? Most argue ‘no’… but some cells have surprizingly specific responses

11. Stimuli used by Baylis, Rolls, and Leonard (1991)

12. Responses of Four Neurons to Face and Non-Face Stimuli in Previous Slide

13. Responses to various stimuli by a neuron responding to a Tabby Cat (Tanaka et al, 1991)

14. The Infamous ‘Jennifer Aniston’ Neuron

15. A ‘Halle Barry’ Neuron

16. Outline of Lecture Neurons: Structure and Physiology Neurons and The Content of Experience  How Neurons Make Inferences  And how these capture features of Bayes Rule Integration of Information in Neurons and in Perception

17. The Key Idea We treat the firing rate of a neuron as corresponding to the posterior probability of the hypothesis for which the neuron stands. If the excitatory inputs to a neuron correspond to evidence that supports the hypothesis for which the neuron stands And the inhibitory inputs correspond to evidence that goes against the hypothesis for which the neuron stands And if the baseline firing rate of the neuron reflects the prior probability of the hypothesis for which the neuron stands And all elements of the evidence are conditionally independent given H. THEN the firing rate of the neuron can represent the posterior probability of the hypothesis given the evidence. Neuron i Input from neuron j w ij

18. Unpacking this idea It is common to consider a neuron to have an activation value corresponding to its instantaneous firing rate or p(spike) per unit time. The baseline firing rate of the neuron is thought to depend on a constant background input called its ‘bias’. When other neurons are active, their influences are combined with the bias to yield a quantity called the ‘net input’. The influence of a neuron j on another neuron i depends on the activation of j and the weight or strength of the connection to i from j. Note that connection weights can be positive (excitatory) or negative (inhibitory). These influences are summed to determine the net input to neuron i: net i = bias i +  j a j w ij where a j is the activation of neuron j, and w ij is the strength of the connection to unit i from unit j. Note that j ranges over all of the units that have connections to neuron i. Neuron i Input from neuron j w ij

19. How a Neuron’s Activation can Reflect P(H|E) The activation of neuron i given its net input net i is assumed to be given by: a i = exp(net i ) 1 + exp(net i ) This function is called the ‘logistic function’ (graphed at right) Under this activation function: a i = P(H i |E) iff a j = 1 when E j is present, 0 when E j is absent; w ij = log(P(E j |H)/P(E j |~H) bias i = log(P(H)/P(~H)) In short, idealized neurons using the logistic activation function can compute the probability of the hypothesis they stand for, given the evidence represented in their inputs, if their weights and biases have the appropriate values, and the elements of the evidence are conditionally independent given H. aiai net i

20. Math Supporting Above Statements Bayes Rule with two conditionally independent sources of information: Divide through by: And let: We obtain: This is equivalent to: And more generally, when {E} consists of multiple conditionally independent elements E j :

21. Choosing between N alternatives Often we are interested in cases where there are several alternative hypotheses (e.g., different directions of motion of a field of dots). Here we have a situation in which the alternatives to a given H, say H 1, are the other hypotheses, H 2, H 3, etc. In this case, the probability of a particular hypothesis given the evidence becomes: P(H i |E) = p(E|H i )p(H i )  i’ p(E|H i’ )p(H i’ ) The normalization implied here can be performed by computing net inputs as before but now setting each unit’s activation according to: a i = exp(net i )  i’ exp(net i’ ) This normalization effect is approximated by lateral inhibition mediated by inhibitory interneurons (shaded unit in illustration). H E

22. Outline of Lecture Neurons: Structure and Physiology Neurons and The Content of Experience How Neurons Make Inferences And how these capture features of Bayes Rule  Integration of Information in Neurons and in Perception

23. ‘Cue’ Integration in Monkeys Saltzman and Newsome (1994) combined two cues to the perception of motion: –Partially coherent motion in a specific direction –Direct electrical stimulation They measured the probability of choosing each direction with and without stimulation at different levels of coherence (next slide).

24. Model used by S&N: S&N applied the model we have been discussing: P i = exp(net i )/  i’ exp(net i’ ) Where P i represents probability of responding in direction i net i = bias i + w ie e +w ij v j w ie = effect of microstimulation on neurons representing percept of motion in direction i e = 1 if stimulation was applied, 0 otherwise W ij = effect of visual stimulation in direction j v j = strength of motion in direction j Visual Input Electrical Input

25. Evidence for the Model Effect of electrical stimulation is absent if visual motion is very strong, but is considerable if visual motion is weak (below). Responses aren’t just averages, but correctly reflect how different sources of evidence should combine, as per the model equation (right) Open circles above show effect of presenting visual stimulation at 90 o (using an intermediate coherence level) together with electrical stimulation favoring the 225 o position. Dip between peaks rules out simple averaging of the directions cued by visual and electrical stimulation but is ~consistent with model predictions (filled circles).

26. Summary: The Mechanism of Unconscious Perceptual Inference Neurons (or populations of neurons) can represent perceptual hypotheses at different levels of abstraction and specificity Connections among neurons can code conditional relations among hypotheses. –Excitation and Inhibition code p(E|H)/p(E|~H) –Lateral inhibition codes mutual exclusivity Propagation of activation produces results corresponding approximately to Bayesian inference. The resulting activity incorporates inferential processes that may alter our phenomenal experience.