Optimality in Motor Control By : Shahab Vahdat Seminar of Human Motor Control Spring 2007

Agenda  Optimal Estimation  Optimal Control  Proposed Model

Optimality  Wolpert, D. M., Ghahramani, Z. & Jordan, M. I. An internal model for sensorimotor integration. Science (1995).  Van Beers RJ, Baraduc P & Wolpert DM. Role of uncertainty in sensorimotor control. Transactions of the Royal Society (2002)  Emanuel Todorov. Optimality principles in sensorimotor control. NatureNeuroscience (2004)  Emanuel Todorov Optimal Control Theory. Bayesian Brain, Doya, K. (ed), MIT Press (2006)

Kalman Filter State-space model is described with these equations:

 The prediction step consists of two calculations: State estimate propagation: Kalman Filter Error covariance propagation:

 The updating step consists of three calculations Kalman Filter Kalman gain matrix: State estimate update: Error covariance update:

For controlling a goal-directed arm movement, there are three sources of noise :  (i) noise in the sensory signals that limits perception,  (ii) noise in the motor commands, leading to inaccurate movements  (iii) sensorimotor noise, which origins from inaccuracies in the forward model and causes noisy predicted location of the body during movement. Therefore, the time-varying Kalman gain is used for minimizing the effect of these noises and uncertainty in the overall estimate. Sources of Noise

Kalman Filter: Sensorimotor Integration

Sensorimotor Integration When we move our arm in darkness, we may estimate the position of our hand based on three sources of information: proprioceptive feedback. a forward model of how the motor commands have moved our arm. by combining our prediction from the forward model with actual proprioceptive feedback. Experimental procedures: Subject holds a robotic arm in total darkness. The hand is briefly illuminated. An arrow is displayed to left or right, showing which way to move the hand. In some cases, the robot produces a constant force that assists or resists the movement. The subject slowly moves the hand until a tone is sounded. They use the other hand to move a mouse cursor to show where they think their hand is located.

Experiment: Sensorimotor Integration

Optimal Control Bellman equations:

Continuous control: Hamilton-Jacobi-Bellman equations:

Deterministic control: Pontryagin’s maximum principle Hamiltonian:

Linear-quadratic-Gaussian control: Riccati equation:

Optimal control as a theory of biological movement state equations:

Optimal control as a theory of biological movement

Open-Loop versus Close-Loop Optimal Controller  Feed forward optimality models explain some of the classical motor properties (bell shaped profiles, etc) Harris & Wolpert, Min. Variance Flash and Hogan, Min. Jerk  Task constraints and motor noise combine to determine optimal motor plans  Humans use continuous visual feedback  Noise in the sensory system very accurately predicts how people use feedback  Task constraints may also impact feedback control strategies

Schematic illustration of open- and closed-loop optimization. (a) The optimization phase, which corresponds to planning or learning, starts with a specification of the task goal and the initial state. Both approaches yield a feedback control law, but in the case of open-loop optimization, the feedback portion of the control law is predefined and not adapted to the task.

(b) Either feedback controller can be used online to execute movements, although controller 2 will generally yield better performance. The estimator needs an efference copy of recent motor commands in order to compensate for sensory delays. Note that the estimator and controller are in a loop; thus they can continue to generate time-varying commands even if sensory feedback becomes unavailable. Noise is typically modeled as a property of the sensorimotor periphery, although a significant portion of it may originate in the nervous system.

Proposed Model: Optimal Primitive State Prediction Force fields as primitives for internal models:

Proposed Model: Optimal Primitive State Prediction Estimation and Control Equations:

Proposed Model: Optimal Primitive State Prediction

Primitive modular representation of the cerebellum