2. Neural Networks: Part 2 Sensory Motor Integration I. Sensory-motor (S-M) Coordination Problem II. Physiological Foundations III. S-M Computation: Tensor Theory (optional)

4. I. Sensory-motor Coordination Problem

5. ( , ) ?=? f( ,  ) Problem to be solved: Given the representation of the target object in visual space, specify the arm position in motor space whether the tip of the arm touches the object.

6. Correspondence Between Sensory Space and Motor Space

9. Projection of Sensory onto Motor Space Non-orthogonal nature of the representations

10. II. Physiological Foundations

11. Superior Colliculus (SC) (deformed topographic map of visual field)

12. Cerebellum (translate topo map of SC into motor coordinate)

13. Sensory-Motor Integration

14. Cerebella Network

15. S-M Computation: Matrix Computation? B = FA where B = ( , ) F = (f ik ), 2x2 matrix A’ = ( ,  ) No, it turns out to perform tensor computation instead. Note: Both types of computation (matrix and tensor) can be represented as neural networks. III. S-M Computation: Tensor Theory (optional)

16. Matrix multiplication in Cerebellum??

17. S-M Transformation: General Case (Question) How about the S-M transformation from a sensory space of n dimensions to a motor space of m dimensions where n and m are different and both greater than 2? (Answer) Tensor Hypothesis The transformation can be represented by a covariant metric tensor (Pellionisz & Llinas, 1985)

18. What Is Tensor? A tensor is a set of numbers specifying relations that exist between two representations of the same object using different, possibly non-orthogonal & over-complete, coordinate systems.

19. Example of Hyper-dimensional Sensory Coordinate System

20. Eye Muscle Activities as Sensory Inputs Note the non-orthogonal (over-complete, i.e., non-unique) nature of the motor system

21. Arm Muscle Activities as Motor Outputs

22. Transformation of Visual cortex activities into arm muscle activities

23. Cerebella Network

24. Neural Circuit: An Example

25. Summary Tensor Theory (Hypothesis): 1. A sensory input is represented by a covariant vector, a motor output by a contravariant vector, and the transformation between them by a covariant metric tensor. 2. In the brain the metric tensor is implemented by a matrix in a neuronal network. (Pellionisz & Llinas, 1985)

26. Tensor Equation for S-M Integration e n = g nk ·i k (n=1,…,N; k=1,…,M) I = (i 1, i 2, …, i M ): Representation in sensory space E =(e 1, e 2, …, e N ) : Representations in motor space G = (g nk ): Tensor that relates I to E Q: What is the object that the above tensorial system purports to represent?

27. Tensor Calculation in the Cerebellum