1. Robots that Work in Collaboration with People Guy Hoffman and Cynthia Breazeal Robotic Life Group MIT Media Laboratory Cambridge, MA, U.S.A

2. Robots learning from us and working with us

3. To be intuitive, efficient and enjoyable these robots must understand and exhibit social skills

4. The goal: working shoulder-to- shoulder with a robot

5. Most HRI work: robots as tools

6. Joint Intention Theory Collaborative Plans  Joint Intention Grosz (1996), Searle (1990) Shared Cooperative Activity, Teamwork Bratman (1992), Cohen & Levesque (1991)

7. Shared Cooperative Activity Meshing Subplans Responsiveness Commitment Support Communication

8. Goals and Collaboration

9. Leonardo Demo

10. So what will it take to collaborate with a robot?

11. Q & A

12. Additional Slides

13. Recent Related Work in HRI Robonaut (2003) Collaboration with Teleoperated Robots Perzanowsky et al. (2001) Natural, Multi-modal Interface for Robot Control Fong et al. (2001) “Collaborative Control” - Using Humans as Source of Information Kimura et al. (1999) Vision-Based Teamwork for a Robotic Arm, Without Social Cues Traum et al. (2003) Virtual Teams Within a Command Structure

14. Social Skills Human Robot Social Function “Let’s do task 5” Looks at the Buttons Leo acknowledges that he understands the task, and visibly establishes mutual belief on the task’s initial conditions Points to Himself He can do the first (and only) part of the task and suggests doing so “OK, you go!” Presses the red button, looking at it Looking away from the partner while operating establishes turn taking boundaries

15. Social Skills Human Robot Social Function Looks back at his partner Gaze shift is used to signal end of turn Nods shortly Acknowledgement signals robot’s perceived end of task. “I’ll go” “Let’s do task 5” Looks at the buttons, points to himself As before Presses the red button Looks at the red button Acknowledges partner’s action, maintaining mutual belief with regard to the task state

16. Social Skills Human Robot Social Function Nods shortly Communicates robot’s perceived end of task. “Let’s do task 5” Looks at the buttons As before Assessing his capabilities, the robot requests support in the task execution Moves the red button out of the robot’s reach Looks at the red button Acknowledges partner’s action, maintaining common ground. Looks back at the human partner; extends his arms in “Help me” gesture.

17. Social Skills Human Robot Social Function “Let’s do task 1” Looks at the buttons, points to himself As before Presses the red button Looks at the red button; looks back at the human partner; nods shortly. Glance acknowledges partner’s action; nod creates mutual belief as to the task’s completion. “OK, you go!” Presses the red button, looking at it Looking away from the partner while operating establishes turn taking boundaries

18. Social Skills Human Robot Social Function At the same time, presses the green button Looks at the green button; looks back at the human partner; nods shortly. Glance acknowledges partner’s simultaneous action; nod creates mutual belief as to the task’s completion. “Let’s do task 7” (unclear) Leans forward, perking one ear Communicates misunderstanding, cues human to repeat the request. “Let’s do task 7” Cocks head, shrugs Communicates verbal understanding, but inability to complete the request. Cues human to adjust the request (in this case, Leo does not know task 7)