Neurodynamics and the Mind Włodek Duch Department of Informatics, Nicolaus Copernicus University, Torun, Poland, School of Computer Engineering, NTU, Singapore Google: W. Duch IJCNN 2011

PlanPlan Dan’s question: what have we learnt about ourselves doing neural modeling? More general: What do we know about ourselves? How can we know it? 1. Geometric model of mind: mind space and other attempts. 2. Is phenomenology possible? 3. Example: imagery agnosias. 4. Spontaneous thoughts and neuron fatigue. 5. Imagery, creativity, insight, expertise …

P-spacesP-spaces Psychological spaces – to see is to understand! K. Lewin, The conceptual representation and the measurement of psychological forces (1938), cognitive dynamic movement in phenomenological space. George Kelly (1955), personal construct psychology, geometry of psychological spaces as alternative to logic. A complete theory of cognition, action, learning and intention. P-space: region in which we may place and classify elements of our experience, constructed and evolving, „a space without distance”, divided by dichotomies. P-spaces (R. Shepard ): minimal dimensionality distances that monotonically decrease with increasing similarity (multi-dimensional non-metric scaling).

Static Platonic model Newton: space-time is an arena for physical events. Psychological spaces should be an arena for mental events. Goal: integrate neural and behavioral information in one model, connect psychology and neuroscience, create mind model at intermediate level. Static version: short-term response properties of the brain, behavioral (sensomotoric) or memory-based (cognitive). Applications: object recognition, category formation in low-dimensional psychological spaces, models of mind. Approach: simplify neural dynamics, find invariants (attractors), characterize them in psychological spaces; use behavioral data, represent them in psychological space.

From neurodynamics to P-spaces. Modeling input/output relations with some internal parameters. W. Freeman: model of olfaction in rabbits, 5 types of odors, 5 types of behavior, very complex model in between. Attractors of dynamics in high-dimensional space => via fuzzy symbolic dynamics allow to define probability densities (PDF) in feature spaces. Mind objects - created from fuzzy prototypes/exemplars.

Human categorization How do we discretize percepts, creating basis for symbolic communication? Multiple brain areas involved in different categorization tasks. Classical experiments on rule-based category learning: Shepard, Hovland and Jenkins (1961), replicated by Nosofsky et al. (1994). Problems of increasing complexity; results determined by logical rules. 3 binary-valued dimensions: shape (square/triangle), color (black/white), size (large/small). 4 objects in each of the two categories presented during learning. Type I - categorization using one dimension only. Type II - two dim. are relevant, including exclusive or (XOR) problem. Types III, IV, and V - intermediate complexity between Type II - VI. All 3 dimensions relevant, "single dimension plus exception" type. Type VI - most complex, 3 dimensions relevant, enumerate, no simple rule. Difficulty (number of errors made): Type I < II < III ~ IV ~ V < VI For n bits there are 2 n binary strings 0011…01; how complex are the rules (logical categories) that human/animal brains still can learn?

Canonical dynamics. What happens in the brain during category learning? Complex neurodynamics simplest, canonical dynamics. For all logical functions one may write corresponding equations. For XOR (type II problems) equations are: Corresponding feature space for relevant dimensions A, B

Inverse based rates. Relative frequencies (base rates) of categories are used for classification: if C is 3 times as coomn as R, and C is associated with (PC, I) symptoms then PC => C, I => C. Predictions contrary to the base: inverse base rate effects (Medin, Edelson 1988). Although PC + I + PR => C (60%) PC + PR => R (60%) Basins of attractors - neurodynamics; PDFs in P-space {C, R, I, PC, PR}. Psychological interpretation (Kruschke 1996): PR is attended to because it is a distinct symptom, although PC is more common. PR + PC activation leads more frequently to R because the basin of attractor for R is deeper.

Feature Space Mapping. FSM (Duch 1994) - neurofuzzy system for modeling PDFs using separable transfer (membership) functions. Categorization (classification), extraction of logical rules, decision support. Set up (fuzzy) facts explicitly as dense regions in the feature space; Initialize by clusterization - creates rough PDF landscape. Train by tuning adaptive parameters P; novelty criteria allow for creation of new nodes as required. Self-organization of G(X;P) = prototypes of objects in the feature space. Recognition: find local maximum of the F(X;P) function.

Trajectories for psycholinguistics P.McLeod, T. Shallice, D.C. Plaut, Attractor dynamics in word recognition: converging evidence from errors by normal subjects, dyslexic patients and a connectionist model. Cognition 74 (2000) New area in psycholinguistics: investigation of dynamical cognition, influence of masking on semantic and phonological errors.

Platonic mind model. Feature detectors/effectors: topographic maps. Objects in long-term memory (parietal, temporal, frontal): local P-spaces. Mind space (working memory, prefrontal, parietal): construction of mind space features/objects using attentional mechanisms.

More connections AI: problem spaces - reasoning, problem solving, SOAR, ACT-R, little work on continuous mappings (MacLennan) instead of symbols. Philosophy: P. Gärdenfors, Conceptual spaces, R.F. Port, T. van Gelder, ed. Mind as motion (MIT Press 1995) Linguistics/CS: G. Fauconnier, Mental Spaces (Cambridge U.P. 1994). Mental spaces and non-classical feature spaces. M. Turner, Conceptual blending, visual blending, metaphore. J. Elman, Language as a dynamical system; J. Feldman neural basis; Stream of thoughts, sentence as a trajectory in P-space. Psycholinguistics: T. Landauer, S. Dumais, Latent Semantic Analysis, Psych. Rev. (1997) Semantic for 60 k words corpus requires about 300 dim.

Are geometric models possible? Why geometric models are so hard? Too many dimensions to consider, so it is hard to see what happens. Brains learn to estimate similarity, only a few dimensions may be related to perception-based experience. Experience cannot be decomposed into components, in fact we are not able to describe our own experience – no phenomenology to model. Eric Schwitzgabel, Russell T. Hurlburt, Describing Inner Experience? Proponent Meets Skeptic, MIT Press (2007). Eric Schwitzgabel, Perplexities of Consciousness, MIT Press (2011). Can I be wrong about my own qualia? Yes! Introspection proved to be of little use in the past. We are not sure if we use mental images in thinking. Do dreams are in color-sound? Not always clear. How do we know what is tactile, auditory, visual? What do I really know when I cannot recall image and name, but have a feeling that I know what it is about?

Can’t find the reasons for may actions 8 Israeli judges, each with 20 years of experience, frequency of granting the parole for prisoners, based on 1000 decisions. (S. Danziger, ). People asked where is the Valentine street were twice as likely to help one minute after the question was asked than people asked about Martina Street (Baumeister et. al, Personality and Social Psychology Bulletin 2009). Dijksterhuis, Nordgren, Unconscious Thought Theory, Perspectives on Psych. Science Nijmegen Unconscious Lab. When judges are hungry chances drop to zero! Do we think with our stomachs? Not knowing the reasons for our decisions does not mean that reasons do not exist. Simple priming has always sn influence.

Imagery and brains How and where are mental images formed? Borst, G., Kosslyn, S. M, Visual mental imagery and visual perception: structural equivalence revealed by scanning processes. Memory & Cognition, 36, , Borst, G., Kosslyn, S. M, Visual mental imagery and visual perception: structural equivalence revealed by scanning processes. Memory & Cognition, 36, , The present findings support the claim that image representations depict information in the same way that visual representations do. Cui, X et al. (2007) Vividness of mental imagery: Individual variability can be measured objectively. Vision Research, 47, Cui, X et al. (2007) Vividness of mental imagery: Individual variability can be measured objectively. Vision Research, 47, Reported Vividness of Visual Imagination (VVIQ) correlates well with the early visual cortex activity relative to the whole brain activity measured by fMRI (r=-0.73), and the performance on a novel psychophysical task. Findings emphasize the importance of examining individual subject variability. Poor perceptual imagery: why? Weak top-down influences? Unable to draw from memory, describe details, faces, notice changes, etc.

Visual top-down Normal perception requires top-down influences to form expectations. What if PC/FC feedback connections to visual/auditory areas are weak? This does not qualify as agnosia, but is a kind of imagery agnosia, a syndrom that has not yet been clearly identified! How will the weak top-down connections in visual modality manifest? Attention problems? Only if signals are very weak (ex. in poor lighting conditions) object recognition may be impaired. Other symptoms: poor visual imagination, memory for visual features, inability to draw from memory, recall and describe faces and objects, notice changes, slow in making puzzles, difficulty to see 3D magic eye pictures, perhaps more introvert? More conceptual than perceptual thinking … recognition memory may work fine. At PC/FC level less interferences from sensory areas, so imagination, creativity, reasoning are fine, perhaps even better than average.

Imagery Agnosia New branch of neuropsychology: imagery agnosias. Classical agnosias ~30 major types: alexia, akinetopsia, alexithimia, many visual types: prosopagnosia, simultanagnosia, semantic agnosia, form, color … Little access to perceptual imagery in visual, auditory, tactile or gustatory mode. Without internal feedback the only way to learn about plans formed by the brain is to act and observe results: trying to play an instrument in this condition is like blindsight, maneuvering blindly in the auditory space, without the ability to imagine results of next move (hitting piano key). Learning to play music without imagery is difficult – how far can one go? Which key do I have to press if I have no idea how it will sound like? Recognition memory is fine, but it is difficult to repeat or remember simple melodies (memory-motor map). No problem to read & improvise music, higher cognition is fine. Conscious mental rehearsal is not possible. Immediate feedback may help?

The Listener James C. Christensen

Model of reading & dyslexia Learning: mapping one of the 3 layers to the other two. Fluctuations around final configuration = attractors representing concepts. How to see properties of their basins, their relations? Emergent neural simulator: Aisa, B., Mingus, B., and O'Reilly, R. The emergent neural modeling system. Neural Networks, 21, , layer model of reading: orthography, phonology, semantics, or distribution of activity over 140 microfeatures of concepts. Hidden layers in between.

Attractors Attention results from: inhibitory competition, bidirectional interactive processing, multiple constraint satisfaction. Basins of attractors: input activations {LGN(X)}=> object recognition Normal case: relatively large, easy associations, moving from one basin of attraction to another, exploring the activation space. Without accommodation (voltage-dependent K + channels): deep, narrow basins, hard to move out of the basin, associations are weak. Accommodation: basins of attractors shrink and vanish because neurons desynchronize due to the fatigue; this allows other neurons to synchronize, leading to quite unrelated concepts (thoughts).

Recurrence plots Same trajectories displayed with recurrence plots, showing roughly 5 larger basins of attractors and some transient points. Starting from the word “flag”, with small synaptic noise (var=0.02), the network starts from reaching an attractor and moves to another one (frequently quite distant), creating a “chain of thoughts”.

Long-term activity Same trajectories displayed with recurrence plots, showing roughly 5 larger basins of attractors and some transient points. A longer “chain of thoughts” starting from the word “flag” and going for 2000 iterations, with small synaptic noise (var=0.02), the network starts from reaching an attractor and moves to another one (frequently quite distant).

Distances from words Starting from the word “flag”, with small synaptic noise (var=0.02), the network starts from reaching an attractor and moves to another one (frequently quite distant), creating a “chain of thoughts”, here shown using distances to known centers of basins of attractors for each trained word.

Normal-ADHD All plots for the flag word, different values of b_inc_dt parameter in the accommodation mechanism. b_inc_dt = 0.01 & b_inc_dt = 0.02 b_inc_dt = time constant for increases in intracellular calcium which builds up slowly as a function of activation.

Normal-Autism All plots for the flag word, different values of b_inc_dt parameter in the accommodation mechanism. b_inc_dt = 0.01 & b_inc_dt = b_inc_dt = time constant for increases in intracellular calcium which builds up slowly as a function of activation.

How to become an expert? Textbook knowledge in medicine: detailed description of all possibilities. Effect: neural activation flows everywhere and correct diagnosis is impossible. Correlations between observations forming prototypes are not firmly established. Expert has correct associations. Example: 3 diseases, clinical case description, MDS description. 1) System that has been trained on textbook knowledge. 2) Same system that has learned on real cases. 3) Experienced expert that has learned on real cases. Conclusion: abstract presentation of knowledge in complex domains leads to poor expertise, random real case learning is a bit better, learning with real cases that cover the whole spectrum of different cases is the best. I hear and I forget. I see and I remember. I do and I understand. Confucius, -500 r.

Summary How can we known what happens in our brain? Sometimes we have to act and observe our action to understand it. Internal qualia dimensions are not easy to define, they may be related to similarity to complex states rather than analysis of separate sensory signals. Essential failure of introspection to provide first person perspective. In many situations it seems impossible to characterize mental content. Imagery agnosia (visual, auditory, other) may become a new area in neuropsychology, activation of subareas lead to strange inner experience. Enhancing imagery using stochastic resonance has been proposed to test this conjecture. Visualization in P-spaces helps to create new language to describe behavior, linking mental and neural events. Visualization of trajectories for words in the semantic space may help to construct models of illustrating memetic processes. Probabilistic landscape of mental spaces changes rapidly in time, spontaneous thought are “weakly linked” to previous thoughts.

Phantomology: the virtual reality of the body. Theoretical aspects and hands-on demonstrations (co-organized with Dept. of Neurology, University Hospital Zurich), with Olaf Blanke, Peter Brugger, Henrik Ehrsson, Kevin O’Regan and others ( ). Brain and music, May 2012 Crossmodality and synesthesia (2012)

Thank you for synchronizing your neurons and lending your ears Google: W Duch => Papers