Neural Networks: Part 2 Sensory Motor Integration I. Sensory-motor (S-M) Coordination Problem II. Physiological Foundations III. S-M Computation: Tensor Theory (optional)

I. Sensory-motor Coordination Problem

( , ) ?=? f( ,  ) Problem to be solved: Given the representation of the target object in visual space, specify the arm position in motor space whether the tip of the arm touches the object.

Correspondence Between Sensory Space and Motor Space

Projection of Sensory onto Motor Space Non-orthogonal nature of the representations

II. Physiological Foundations

Superior Colliculus (SC) (deformed topographic map of visual field)

Cerebellum (translate topo map of SC into motor coordinate)

Sensory-Motor Integration

Cerebella Network

S-M Computation: Matrix Computation? B = FA where B = ( , ) F = (f ik ), 2x2 matrix A’ = ( ,  ) No, it turns out to perform tensor computation instead. Note: Both types of computation (matrix and tensor) can be represented as neural networks. III. S-M Computation: Tensor Theory (optional)

Matrix multiplication in Cerebellum??

S-M Transformation: General Case (Question) How about the S-M transformation from a sensory space of n dimensions to a motor space of m dimensions where n and m are different and both greater than 2? (Answer) Tensor Hypothesis The transformation can be represented by a covariant metric tensor (Pellionisz & Llinas, 1985)

What Is Tensor? A tensor is a set of numbers specifying relations that exist between two representations of the same object using different, possibly non-orthogonal & over-complete, coordinate systems.

Example of Hyper-dimensional Sensory Coordinate System

Eye Muscle Activities as Sensory Inputs Note the non-orthogonal (over-complete, i.e., non-unique) nature of the motor system

Arm Muscle Activities as Motor Outputs

Transformation of Visual cortex activities into arm muscle activities

Cerebella Network

Neural Circuit: An Example

Summary Tensor Theory (Hypothesis): 1. A sensory input is represented by a covariant vector, a motor output by a contravariant vector, and the transformation between them by a covariant metric tensor. 2. In the brain the metric tensor is implemented by a matrix in a neuronal network. (Pellionisz & Llinas, 1985)

Tensor Equation for S-M Integration e n = g nk ·i k (n=1,…,N; k=1,…,M) I = (i 1, i 2, …, i M ): Representation in sensory space E =(e 1, e 2, …, e N ) : Representations in motor space G = (g nk ): Tensor that relates I to E Q: What is the object that the above tensorial system purports to represent?

Tensor Calculation in the Cerebellum