Bayesian goal inference in action observation by co-operating agents EU-IST-FP6 Proj. nr Raymond H. Cuijpers Project: Joint-Action Science and Technology (JAST) Nijmegen Institute of Cognition and Information Radboud University Nijmegen The Netherlands

Outline About Joint Action The problem of action observation –Simulation theory –Goal inference –Ingredients functional model Functional model of goal inference –Scenario –Architecture Simulation results Conclusions

Joint action Multiple levels of co-ordination –Kinetic: force, timing –Kinematic: speed, trajectory –Action level: what to do? –Goal level: for what purpose? –Reasoning: how to achieve destination? Actions of co-actors typically differ –Action observation –Anticipation of behaviour of co-actor Common (ultimate) goal –Action sequences –(immediate) action goal inference About Joint Action

The problem of action observation

How can we infer the observed action? Simulation theory: use own motor system to simulate actions of other Examples: Motor control theory: –Forward modelling: Predict consequences of actions –Action observation: predict observed action from action repertoire Robotics: –Direct mapping of observed joint angles on those of own action repertoire Problem: Requires similar effectors and kinematics Perception depends on viewpoint The problem of action observation

Mirror Neurons Rizzolatti, Fadiga, Gallese, & Fogassi (1996). Mirror neurons fire both during observation and execution of similar actions Evidence for simulation theory  Shared resources for performing and observing actions Ideomotor compatibility Brass, Bekkering, Wohlschlaeger, & Prinz (2000). Lift finger indicated by symbol Response is faster when performing congruent actions  Action system is used in action observation The problem of action observation

How can we recognise a dog catching a frisbee? Different body Observed effector differs from own effector (mouth vs. hand) Different kinematics  Direct mapping of joint angles is impossible  Forward modelling is impossible Inference must occur at more abstract level: goal inference The problem of action observation

Imitation of 14-months infants Gerely, Bekkering, & Kiraly (2002). Nature. Infants imitate with hands when the actor’s hands were occupied Evidence for goal inference Hands occupiedHands free Imitate using handsImitate using head  Imitate action goals rather than the effector The problem of action observation

Evidence for goal inference Mirror neurons Fogassi, Ferrari, Gesierich, Rozzi, Chersi & Rizzolatti (2005). Science 308: Firing rate during grasping depends on subsequent movement Activity is selectively tuned to the action goal (=destination of food) The problem of action observation

Ingredients for functional model Viewpoint invariance –Use viewpoint independent measures (distance, colour) Infer action goals (=intended state change of the world) –make decision at goal level –consistent with final goal state of a sequence of acts Use your own action system for observation –Use own action repertoire –Use own preferences –Use own task knowledge assume common The problem of action observation

Functional model of goal inference during action observation Cuijpers RH, Van Schie HT, Koppen M, Erlhagen W and Bekkering H (2006) Goals and means in action observation: a computational approach. Neural Networks 19:

Sequence of primitive motor acts (screw nut, put bolt through hole) Observable current state and final goal state (final construction) Shared task knowledge (action repertoire, action goals) Not shared: Action sequence, viewpoint and personal preferences Two agents co-operatively build a model from Baufix building blocks ? Initial stateFinal goal state Model of action goal inference

ActorObserver Model of action goal inference (Cuijpers et al., 2006) Belief that goal is red-bolt-screwed-in-green-nut Belief that action is to screw red bolt in green nut Likelihood that hand moves to red bolt Belief that hand moves to red bolt Observation Decision marginalisation rule Bayes rule Model of action goal inference

Two fundamental processes Turn evidence into beliefs (Bayes’ rule) Belief propagation (marginalisation rule) Model of action goal inference Pr(screw red bolt in green nut) = Pr( screw red bolt in green nut | if target is n ) x Pr( target is n )  Posterior belief Evidence Personal preference Pr( red bolt | observ. ) ~ Pr( observ.| if target is red bolt ) x Pr( red bolt ) Action level Knowledge own action repertoire Component level n

Viewpoint invariance Observations depend on viewpoint invariant measures –Distance between effector and target –Rate of distance change Model of action goal inference

Use your own action system Belief propagation uses task knowledge –Components required for each action alternative p(c n |A k ) –Action goal associated to each action alternative p(i  j|A k ) Use personal preferences (priors) –component preferences p(c n ) –Action preferences p(A k ) –Action goal preferences for a given final goal state p(i  j|f) Execution and observation share resources –Task knowledge –Personal preferences Model of action goal inference

Infer action goals rather than means Infer action goal beliefs p(i  j|o t,f) –Consistent with final goal state f Make decision at goal level –Belief in action goal p(i  j|o t, f) > threshold Model of action goal inference

Simulation results

Scenario Joint Task: Actor: Action Goal: bolt through slat Action Alternative: c1+c5 First target: c1 Observer: infer goal c1 c2 c3 c4 c5 Simulation results

c1 c2 c3 c4 c5 Belief component c n is the target p(c n |o t ) Nearby targets are more likely unless movement speed is high Beliefs are biased by personal preferences c 1 correctly identified after 40% of movement time (MT) Belief that goal is red-bolt-screwed-in-green-nut Belief that action is to screw red bolt in green nut Likelihood that hand moves to red bolt Belief that hand moves to red bolt Simulation results

Belief in action alternatives p(A k |o t,f) Only possible actions (task knowledge) Only actions consistent with goal state f (task knowledge) Action alternatives with nearby targets are more likely c1 c2 c3 c4 c5 Impossible! Belief that goal is red-bolt-screwed-in-green-nut Belief that action is to screw red bolt in green nut Likelihood that hand moves to red bolt Belief that hand moves to red bolt Simulation results Inconsistent

Belief in action goals p(i  j|o t,f) Inconsistent action goals are suppressed (task knowlegde) Correct action goal is inferred after 23% of MT The correct action goal is inferred before the action or the target component c1 c2 c3 c4 c5 Belief that goal is red-bolt-screwed-in-green-nut Belief that action is to screw red bolt in green nut Likelihood that hand moves to red bolt Belief that hand moves to red bolt Simulation results

Conclusion We made a functional model that captures behavioural and neurophysiological findings on action observation Missing knowledge about the co-actor is replaced by task knowledge from the observer’s own action repertoire To inference process is driven by the likelihood of observed movements and is biased by personal preferences Action planning is driven by the intended goal and by personal preferences As a consequence imitation need not involve the same effector (imitation) Actions are not directly mapped onto the observer’s repertoire. Consequently, complementary actions can be as fast as imitative actions in a joint action context

Thank you for your attention!

p(c n |o t ) ~ p(o t |c n ) p(c n ) p(i  j|A k,f) = p(A k |i  j) p(i  j|f)/ p(A k |f) p(A k |c n ) = p(c n |A k )p(A k )/p(c n ) p(A k |o t ) =  n p(A k |c n ) p(c n |o t ) p(i  j|o t,f) =  k p(i  j|A k,f) p(A k |o t ) Component belief  likelihood, preference Action belief  action knowledge, component belief Goal belief  goal knowledge, action belief