1. Effect of Shared-attention for Human- Robot Interaction Junji Yamato jy@acm.org NTT Communication Science Labs., NTT Corp. Japan Kazuhiko Shinozawa, Futoshi Naya ATR Intelligent Robot and Communication Labs.

2. Aim To build Social Robot/Agent Sub goal To establish Evaluation methods Design guidelines for communication of human-robot/agent

3. To measure the influence of Agent/Robot on users Acceptance ratio of agent/robot recommendation Method

4. Color name selection task Blue or Green? Cobalt green or emerald green? Skin color or KARE-IRO? SUMIRE-IRO or AYAME-IRO? ---- Total:30 questions. (from color name text book) No “ correct ” answer Easy to be influenced

5. Four experiments 1. Compared agent and robot 2. Compared agent and robot in physical world 3. Measured the effect of eye contact 4. Measured the effect of shared-attention Yamato, J., Shinozawa, K., Brooks, R., and Naya, F. Human-Robot Dynamic Social Interaction. NTT Technical Review 1, 6(2003), 37-43. Detailed description of Experiment 1 and 2 Available on-line http://www.ntt.co.jp/tr/ Back number -> Sep. 2003 Shinozawa, K., Naya, F., Yamato, J., and Kogure, K. Differences in Effect of Robot and Screen Agent Recommendations on Human Decision-Making, IJHCS (to appear) Experiment 1, 2, and description of K4(robot)

6. Experiment 1 ： Compare Agent and Robot Conditions: 30 questions, 30 subjects in each group ‐ Same question sequences, same voice, similar gesture Measurement: acceptance ratio, questionnaire AgentRobot AgentRobot

7. Experiment 1: Robot

8. Experiment 1: Result Acceptance ： agent > robot (p<.01) Familiarity ： independent

9. Initial expectation Robot has more influence because it lives in 3D world, same as subjects. agent robot ○ × Gap

10. Experiment 2: Compare in physical world Color plate Button box No recommendation (30 subjects) Recommendation by robot （ 31 subjects ） Recommendation by agent (30 subjects)

11. Experiments

12. Experiment 2: Result selection ratio ： robot > agent ( p < 0.05) robot >> no recommendation ( p < 0.01)

13. Experiment 1 and 2: Results Media world Physical world agent robot Consistency matters. ○ × ○ × Embodiment and communication

14. Why robot is better? Easy to detect gaze – Eye contact – Shared attention/joint attention Measure the effect of eye contact and shared-attention

15. Eye contact was established by face tracking Eye contact time: period that subject looked at robot and robot looked at subject Eye contact time and selection ratio? Two groups (14 subjects each) – Eye contact, and NO eye contact Experiment 3: Effect of eye contact (mutual gaze)

16. Robots

17. Higher selection ratio for eye contact group K4: No E.C. < E.C. (p=0.012) Rabbit: No E.C. < E.C. (p=0.003) Selection ratio

18. Shared attention: – Period that robot looks at an object and subject looks at the same object. (color plate, button box) SA time and selection ratio – Is there correlation? Experiment 4: Effect of shared-attention

19. Robot looks at color plate and button box by prepared program Eye contact established by face tracking Establishing shared- attention Example: video

20. 28 subjects SA time = 51.7 sec (total for 30 questions) – (Longer than in Experiment 3 ) Selection ratio. Average: 0.57 S.D.= 0.14 Some subjects were positive, and others were not. Clear contrast, from the questionnaire. Example: Robot is prompting wrong choice. I feel the robot forced me to select his recommendation (negative). Experimental conditions

21. No correlation SA time and selection ratio Shared-Attention time (count) 50count=1sec. Selection ratio

22. Ego-gram based on transactional analysis Measure three ego-states by questionnaire – CP, NP (critical parent, nurturing parent) – A (adult) – FC, AC (free child, adapted child) TEG (Tokyo Univ. Egogram ） is common in Japan Clustering subjects by TEG (Ego-gram)

23. Strong correlation in SA time and acceptance ratio for high AC (Adapted Child) group High/Low TEG measurement and SA time. CPNPAFCAC Threshold (threshold)1017121412 Selection ratio High group0.5487179490.5644444440.5642857140.6179487180.597777778 Low group0.5866666670.5743589740.5738095240.5266666670.535897436 SAtime 2553.4615382581.9333332575.2857142587.6153852510.866667 2612.2666672588.4615382594.6428572582.6666672670.461538 Correlation fo High - 0.046541377 - 0.035889649 -0.031458210.1361527370.405908084 SAtime and sel ratio Low0.0546489390.109967510.055147415 - 0.078258186 -0.461514856

24. Positive correlation(Speaman’s r=0.51,p=0.051). SA time and selection ratio (high AC & low CP group)

25. High-SA group = high selection ratio (p<0.05) SA time and selection ratio （ high AC group)

26. High AC subject (obedient type) showed positive correlation between SA time and selection ratio. No significant difference between SA time itself and selection ratios for high AC and low AC groups Eye contact and shared-attention promote close communication. Some people like such intimate relation, but others don’t. It depends on the character. SA is effective. Even SA was not “actually” realized. We do not need to develop image understanding technology; we just have to fake it. Result and Discussion