Presentation is loading. Please wait.

Presentation is loading. Please wait.

Learning and Remapping in the Sensory Motor System F.A. Mussa-Ivaldi Northwestern University.

Similar presentations

Presentation on theme: "Learning and Remapping in the Sensory Motor System F.A. Mussa-Ivaldi Northwestern University."— Presentation transcript:

1 Learning and Remapping in the Sensory Motor System F.A. Mussa-Ivaldi Northwestern University

2 In natural arm movements, complex motions of the joints lead to simple hand kinematics From Morasso 1981

3 Is the shape of trajectories the byproduct of dynamic optimization? (Uno, Kawato and Suzuky, 1989)

4 Kinematic Adaptation Flanagan and Rao (1995)

5 Dynamic Adaptation Unperturbed movements Shadmehr & Mussa-Ivaldi, 1994

6 Perturbing field Note: there is no force at rest

7 Adaptation After-effects

8 Transport of a spring-mass object Dingwell, Mah and Mussa-Ivaldi (2004)

9 Post-adaptation kinematics Baseline Initial response

10 The shape of trajectories is not a by-product of dynamic optimization. It reflects and is consistent with the Euclidean properties of ordinary space

11 What is space ? Intuitive Concept Geometry, from a physical standpoint is the totality of the laws according to which rigid bodies mutually at rest can be placed with respect to each other… Space in this interpretation is in principle an infinite rigid body – or skeleton - to which the position of all other bodies is related. A. Einstein, 1949 Less Intuitive Concepts Signals spaces, configuration spaces, state spaces, feature spaces, etc...

12 Ordinary Space Euclid Pythagoras Two equivalent statements x y d

13 Euclidean Symmetry Among all possible norm only the L 2 (Euclidean) norm is invariant under rigid transformations (e.g. rotations, translations). Size does not depend upon position and orientation. Euclidean (L 2 ) Norm Orthogonal transformation

14 Signal spaces Neither the visual nor the motor signals spaces are Euclidean. Yet, we perceive Euclidean symmetry and we coordinate motor commands to control independently movement amplitude and direction

15 It is a common misconception that the visual system provides the primal information about this symmetry. In fact it is likely for the opposite to be true: we learn the properties of space through active perception

16 Is the bias toward Euclidean symmetry a guiding principle for the emergence of new motor programs?

17 The motor system can reassign degrees of freedom to novel tasks Glove Talk. Sidney Fels and Geoffrey Hinton, 1990 VocabularySam-I-am

18 The properties of space are captured by the motor system when forming novel motor programs whose purpose is to control the motion of an endpoint

19 19 SIGNALS (Hand Configuration) 2 Screen Coordinates F.A. Mussa-Ivaldi, S. Acosta, R. Scheidt, K.M. Mosier Remapping Space

20 The task The subject must place the cursor corresponding to the hand inside a circular target Experiment begins with the cursor inside a target. New target is presented – Cursor is suppressed. The subject must try to make a single rapid movement of the fingers so as to place the (invisible) cursor inside the new target and stop. As the hand is at rest after this initial movement, the cursor reappears The subject corrects by moving the cursor under visual guidance inside the target Only the initial open loop movement is analyzed

21 Part 1 Part 2 A B C D Some examples

22 Weird task. Cannot be represented before the experiment Body and task endpoint are physically uncoupled Only feedback pathway is vision Dimensional imbalance Metric imbalance Some observations

23 EnsembleSingle Subject Endpoint error

24 Linearity S E A B Aspect-Ratio: AB/SE Ensemble Single Subject Cursor movements DO NOT become straighter !

25 Extended training 4 consecutive days 2 groups of subjects No-Vision subjects (P2) just engage in the task for 4 days Vision subjects (P3) train with continuous visual feedback trials. But they are tested in the same no-vision condition as the other group. The two groups receive the same amount of training

26 Extended Training Results Both group have the same learning trend. Continuous visual feedback of endpoint movements facilitates the formation of straighter open-loop trajectories

27 Error reduction and straightening of motion are independent processes Straightening of motion is facilitated when vision and movement execution are concurrent

28 The subjects learn to solve an ill-posed inverse problem (from cursor location to hand gesture) and in doing so, they learn the geometrical structure of the map, i.e. a specific partition of degrees of freedom.

29 Decomposition of Glove Signals

30 Task-space and Null-space Motion 19% 24% 20% 23%

31 Importing a metric structure

32 Identify Residual Control Signals (Movements, EEG, EMG, Multiunit…) Form a first vocabulary of commands (faster, slower, right, left, MIDI, etc…) Machine learns to respond appropriately Patient practice and leans new commands Practical Applications: Body-Machine Interfaces (BMI) Interface DUAL LEARNING

33 Sensing garments (Smartex -Pisa) 52 smeared sensors on lycra No metallic wires External connection realized by metallic press stud

34 Calibration: Find optimally controlled DOFs (synergies) by dimensionality reduction

35 Natural Synergies -> Control Parameters A bad map A better map

36 Tonal Spaces

37 The interaction of human and machine learning: Co-Adaptation by Least Mean Squares

38 The interactive LMS procedure - 1 1) Set an arbitrary starting map: Endpoint vector dim(x) =2 Body vector dim(h)=20

39 The interactive LMS procedure - 2 2) Collect movements toward a set of K targets: Targets Movements (endpoint coordinates) Movements (body coordinates)

40 The interactive LMS procedure - 3 3) Calculate the error (in endpoint coordinates) For each movement For a whole test epoch

41 The interactive LMS procedure - 4 4) Calculate the gradient of the error (w.r.t. A) 5) Update the A-matrix in the direction of the gradient 4) Repeat from step 2 until convergence.

42 An Example

43 Conclusion Conclusion Sensory-Motor Cognition IS NOT an oxymoron !

Download ppt "Learning and Remapping in the Sensory Motor System F.A. Mussa-Ivaldi Northwestern University."

Similar presentations

Ads by Google