1. Ali Ghadirzadeh, Atsuto Maki, Mårten Björkman Sept 28- Oct 2 2015. Hamburg Germany Presented by Jen-Fang Chang 1

2. Outline Introduction Proposed method Experiment results Conclusion and Future work 2

3. Introduction Sensorimotor contingencies(SMC) based method Sensory awareness are the result of integrating sensorimotor coupling into the planning system. Grounded directly to the environment, give the flexibility to designs systems with self-learning. 3

4. Visual servoing Visual servoing is an approach to control a robot’s motion using feedback. Unlike tradition methods to design system, there is no need to calibrate the robot by SMC based approach. The task is grounded to the environment, enabling self- learning without intervention. 4

5. Main contribution Eliminate the need for prior knowledge of kinematic or dynamic models of the robot Use forward model to search for proper actions to solve tasks by minimizing a cost function, instead of training a separate inverse model, to speed up training Encode 3D spatial positions of a target project to avoid calibration with external coordinate system 5

6. Related Works Forward model with Distal Supervised learning (DSL) Combined forward-inverse model learning Affordance theory learning Reinforcement learning PILCO 6

7. Proposed method Training a forward model Finding inverse output Gaussian Process regression Kinematic system Visuomotor tasks 7

8. Training a forward model START Target S* cost function initial train forward model with randomly generated action-observation pairs Target state reached? No Yes Gaussian Process regression Find inverse output General form forward model: 8

9. Inverse output The cost function defined as (1) To find the optimum motor command that minimize the cost function, we use gradient based method: (2) Equation (2) can be written as: (3) 9

10. Apply gradient descent repeatedly gets an optimum unbounded value: (4) Insert the optimum value to (5) We can get the motor commands that optimize the cost function 10

11. Gaussian Process regression A non-parametric model where the representation is given by the training samples. The key factor to successfully learn the forward models in real-time. 11

12. Kinematic system The camera joints consist Jnp (neck pan), Jnt (neck tilt) and Jv (Vergence angle) The robot arms joints consist Jsp(shoulder pan), Jsl(shoulder lift), Jar (arm roll) and Je(elbow) The control architect is hierarchical and can be regarded as a look and move architecture A position is defined by camera joints that would be required for the robot to fixate onto the object 12

13. Visuomotor tasks Fixation task Fixate onto an object which can be observed with the maximum overlap between the two camera view The process serves as a mean to probe the 3D position Reaching task Use the information of 3D position and try to reduce the distance between the end-effector and target 13

14. 14 Fixation forward model structure Reaching forward model structure Jv missing

15. Experiment results Learning performance for the training and test phases of the fixation task, Implemented on the PR2 robot. 15 Model training

16. Learning performance for the reaching task, (a) with the same target used for training and test, and (b) for two new target. 16

17. Cost functions evaluation 17

18. 3D position encoding 18 The Euclidean distance between two points in the joint space against the 3D distance in Metric space

19. Tolerance against image distortion The cost functions are given as the average of 10 different trials, performed in the simulation environment. Image with no distortion, moderate(λ= 0.9) and considerable(λ= 0.7) 19

20. Conclusion The most important feature of the proposed method are the real-time learning and the fact that it requires the robot to only have a few interaction with the environment. One key factor to speed up the training is that we applied the forward model to search for the motor commands by minimizing a given cost, instead of training a separate inverse model. 20

21. Future work Control a robot arm while avoiding obstacles Utilize other regression model for different event 21

22. https://www.youtube.com/watch?v=qxDPU-nS0xo https://www.youtube.com/watch?v=9JQOcg3Jcqs 22