Neural Organization and Connectionist Models of Spatial Cognition: A Review Jigar Patel and S. Bapi Raju* University of Hyderabad Spatial Cognition The study of spatial cognition and spatial language focus on the representation and processes that are used for encoding, storing, accessing and referencing spatial knowledge (Ghiselli-Crippa, Hirtle and Munro,1996). These processes enable recall of spatial information, navigation through space, spatial decision making, and for generating spatial descriptions. This poster highlights the following aspects of spatial cognition: –Spatially directed actions, path of motion and manner of motion. –Spatial prepositions, such as above, below, etc. The Neural Organization of Spatial Cognition: –perceptual and conceptual aspects of space have parallel representations in the cortex – Temporal and dorsal regions of cortex. –the perceptual inputs and representations are located posteriorly but, the conceptual systems are located more anteriorly and medially in the cortex (chatterjee 2008). Connectionist models of Spatial Cognition: –Dual-route networks along with simple Elman recurrent networks are used for spatial preposition judgments.(Coventry et al. 2005) Spatial Prepositions Model: –Vision Processing : processes the image sequences at a variety of spatial scales and resolutions for object form and motion features yielding a visual buffer. –Elman Network Module: Consists of predictive time-delay connectionist network referred as Connectionist Perceptual Symbol System Network (CPSSN; Joyce et al., 2003). uses 12x9 activations input (generated by visual processing) and previous state of hidden units to train with backpropagation algorithm. Total 3 networks in model, one with 20 hidden units and two with 5 hidden units. –Dual-route network: A feedforward neural network uses hidden activations from Elman network and linguistic data as input. Trained via error backpropagation algorithm. Task: –Focused on only over/under/above/below prepositions. –Participants needed to rate a picture of 3 different objects (a container, plate/dish/bowl and liquid) together with sentence to describe each picture using Lickert scale where 1 =totally unacceptable and 9 = totally acceptable. –Total 4 experiments: 1) Movies of pouring scenes. 2) compared full movies with the end states. 3) compared end state to an earlier frame in the movie. 4) to test model with experimental data, in this experiment, it was suggested to the participants to mentally ‘play’ with visual scene and auto-generate outcome of the scene to rate linguistic utterance. Results: –The comparison of the subjects’ rating data and the networks’ output of the 4 prepositions resulted in an RMS error of which confirms that the model had predicted very accurately the ratings. The Neural Organization of Spatial Cognition Summary of Chatterjee(2008) Summary of Coventry et al Our Research Plan Experiment methods and Findings –A one-back matching block design in which conditions of interest were “star” figure moving with different manners and paths subjects were to attend path of motion( e.g. figure is moving with relation of some external referent for example, “running across the road”) or to the manner of motion( e.g. figure is moving in referent to itself for example, rotating ). Greater activation for manner than path was seen bilaterally within the posterior middle temporal gyrus (pMTG) while greater activation for path than manner was seen bilaterally in the posterior intraparietal sulcus (IPS,) and posterior middle frontal gyrus (pMFG). The aspects of movements and actions through the space together to form events that fundamentally structure how we think of the world in a dynamic sense. –A testing procedure in which triads of stimuli were presented and subjects matched one of two target stimuli to a test stimulus based on their similarity. For example, in the action condition, subjects might match a picture of digging to shoveling rather than sewing while the object trials showed the pictures of static objects. When people make semantic judgments of actions, they activate visual motion and adjacent areas even though the stimuli under consideration themselves were not moving. –A study using verb triads in which the association was based on manners of motion. For example, ‘‘hopping’’ would be matched to ‘‘skipping’’ rather than ‘‘running.’’ all the choices referred to actions with translation motion. While The areas activated by action words were adjacent to the area specialization in visual motion but they were not identical which supports the idea that motion attributes are processed along the lateral temporal cortex with gradient in which concrete to abstract information is processed along a posterior to anterior ( toward peri-Sylvian cortex) axis. –They contrasted spatial metaphors in sentences like ‘the man fell under her spell’ to literal motion sentences like ‘the child fell under the slide’. They found greater activation in the left inferior frontal cortex and left lateral temporal lobe for spatial metaphors as compared with literal sentences and no differences in homologous areas of the right hemisphere. To develop a dual route model for representing path and manner to replicate behavioral data produced by Chatterjee, The Basic Framework is shown Below. References –Ghiselli-Crippa, T., Hirtle, S.C. and Munro, P. (1996). Connectionist Models in Spatial Cognition. In J. Portugali (ed.), The Construction of Cognitive Maps (pp ). Kluwer Academic Publishers. –Chatterjee A. The Neural Organization of Spatial Thought and Language. Seminars in Speech and Language. 2008, 29: –Coventry, K.R., Cangelosi, A., Rajapakse, R., Bacon, A., Newstead, S., Joyce, D., Richards, L.V.: Spatial prepositions and vague quantifiers: Implementing the functional geometric framework. In: Proceedings of Spatial Cognition Conference (2004) The computational model for the processing of visual scenes and the identification of the appropriate spatial preposition consists of three main modules: –(1) Vision Processing, –(2) Elman Network, –(3) Dual-Route Network. Where ( Dorsal Stream) What ( Ventral Stream) Visual Processing Unit Path of Motion Processing Manner of Motion Processing Dual-Route Network Linguistic Prediction