A.G. Billard, Autonomous Robots Class Spring 2007 Doctoral School – Robotics Program Autonomous Robots Class Spring 2007 Human-Robot Interaction Robots in education, therapy and rehabilitation Aude G Billard Learning Algorithms and Systems Laboratory - LASA EPFL, Swiss Federal Institute of Technology Lausanne, Switzerland

A.G. Billard, Autonomous Robots Class Spring 2007 Overview of the Class 1.Interfaces and interaction modalities (~45 min) 2.User-centered design of social robots (~45 min) + Contest (~15 min) 3.Social learning and skill acquisition via teaching and imitation (~1.15 min) 4.Robots in education, therapy and rehabilitation (~45 min) 5.Evaluation methods and methodologies for HRI research (~30 min) 6.Ethical issues in human-robot interaction research (~15 min) + DISCUSSION (~30 min)

A.G. Billard, Autonomous Robots Class Spring 2007 Current real-life applications of HRI are in Education, Therapeutic and Rehabilitation These applications rely often on toy-like robots that are small, and, as such convey a sense of security and cuteness Robots in education, therapy and rehabilitation

A.G. Billard, Autonomous Robots Class Spring 2007 Robots in Therapy Rationale: Aging population is increasing  loneliness of the elderly Sick child or elderly person in a home  loneliness of the sick  a toy robot replaces a domestic animals in environments where those cannot be accepted (allergies)  petting the toy robot will improve the patient’s mood, fulfill its need to care and be cared for and enhance interaction and communication across patients (a mediator)  a robot allows to follow the patient’s development in longitudinal studies Use for the disabled (autistic children, paraplegics, etc) Use for the elderly

A.G. Billard, Autonomous Robots Class Spring 2007 Robots in Therapy Paro, the Seal Robot (AIST, Japan) 8 actuators; 2 for upper and lower eyelids, 1 for rotation of eyes, 2 for the neck, 1 for each front fin, and 1 for two rear fins. Paro weights about 2.8 kg.

A.G. Billard, Autonomous Robots Class Spring 2007 Robots in Therapy Paro has internal states that correspond to emotions. Each state activity changes as a function of the interactions and otherwise decays with time. Interaction changes the internal states of Paro and, thus, the user creates the character of Paro. Reinforcement learning: positive for stroking, negative for beating.

A.G. Billard, Autonomous Robots Class Spring 2007 Robots in Therapy Tests: 2-15 years old children – hospitalized; Paro robot was presented 3 time a day over the course of 11 days Results: The children’s moods improved on interaction with Paro, encouraging the children to communicate with each other and caregivers. In one striking instance, a young autistic patient recovered his appetite and his speech abilities during the weeks when Paro was at the hospital. In another case, a hopitalized child patient who felt pain when she moved her body, arms, and legs and could not move from her bed. When Paro was given to her, she smiled and was willing to stroke Paro. Paro had a rehabilitative function as well as a mental effect. Wada, K.; Shibata, T.; Saito, T.; Tanie, K, Effects of robot-assisted activity for elderly people and nurses at a day service center, Proceedings of the IEEE, Volume 92, Issue 11, Nov Page(s):1780 – 1788.

A.G. Billard, Autonomous Robots Class Spring 2007 Robots in Therapy Tests: Elderly people with slight dementia; Paro robot was presented 3 day a week over the course of 5 weeks. Results: Paro improved the mood of elderly people and made them more active. Paro encouraged elderly people to communicate, both with one another and with the nursing staff. Urinary tests showed that the ability to recover from stress was improved in the elderly. Wada, K.; Shibata, T.; Saito, T.; Tanie, K, Effects of robot-assisted activity for elderly people and nurses at a day service center, Proceedings of the IEEE, Volume 92, Issue 11, Nov Page(s):1780 – 1788.

A.G. Billard, Autonomous Robots Class Spring 2007 Robots in Therapy Wada, K.; Shibata, T.; Saito, T.; Tanie, K, Effects of robot-assisted activity for elderly people and nurses at a day service center, Proceedings of the IEEE, Volume 92, Issue 11, Nov Page(s):1780 – 1788.

A.G. Billard, Autonomous Robots Class Spring 2007 Robota allows the child to understand that the robot’s movements originate from his own movement (sense of agency) and are limited to a restricted category of movements (intentional selection of his own action ) Robota: A test of understanding self- agency Collaboration with Prof. Jaqueline Nadel Hopital de la Salpetriere, Paris, France Robots for Therapy

A.G. Billard, Autonomous Robots Class Spring 2007 The educator demonstrates The imitation game The autistic child is let free to play with the robot A tool to test the imitation capabilities of autistic children Collaboration with Prof. Jaqueline Nadel Hopital de la Salpetriere, Paris, France Robots for Therapy Billard, A.Billard, A., Robins, B, Dautenhahn, K. and Nadel, J (2006) Building Robota, a Mini-Humanoid Robot for the Rehabilitation of Children with Autism. the RESNA Assistive Technology Journal. Vol. 19, Issue 1.

A.G. Billard, Autonomous Robots Class Spring 2007 Robins, B., Dautenhahn, K., te Boekhorst, R. Billard, A. (2004) Effects of repeated exposure of a humanoid robot on children with autism In S.Keats, J.Clarckson, P. Langdon and P.Robinson (eds), Designing a More Inclusive World. London: Springer Verlag, Would the human-appearance affect the child’s interaction with the robot? A tool to test and train the interactivity of autistic children Children are much more responsive toward non-human looking doll Robots for Therapy

A.G. Billard, Autonomous Robots Class Spring 2007 A tool to test and train the interactivity of autistic children Four autistic children, age 5-10, from the Enhanced Provision unit. Each child participated in about 13 trials on average. Autistic children interact much more with the non-human looking robot Robins, B., Dautenhahn, K., te Boekhorst, R. Billard, A. (2004) Effects of repeated exposure of a humanoid robot on children with autism In S.Keats, J.Clarckson, P. Langdon and P.Robinson (eds), Designing a More Inclusive World. London: Springer Verlag, Robots for Therapy

A.G. Billard, Autonomous Robots Class Spring 2007 A tool to test and train the interactivity of autistic children Collaboration with Prof. Kerstin Dautenhahn, University of Hertfordshire, London, UK Robot as a mediator, an object of joint attention Robins, B et al. (2004) London: Springer Verlag, Robots for Therapy

A.G. Billard, Autonomous Robots Class Spring 2007 Eye-Gaze levels of all children Robins, B., Dautenhahn, K., te Boekhorst, R. Billard, A. Springer Verlag, A tool to test and train the interactivity of autistic children Robots for Therapy

A.G. Billard, Autonomous Robots Class Spring 2007 ROBOTA at the Museum French National Science Museum, « La Cite de l’Espace », Paris, November 2001 – January 2003 Robots for Education

A.G. Billard, Autonomous Robots Class Spring 2007 ROBOTA at the Museum French National Science Museum, Toulouse, May 2004 – December 2006 Fondation Verdan, Lausanne, November 2004-March 2006 Robots for Education

A.G. Billard, Autonomous Robots Class Spring 2007 ROBOTA at the Museum Monitored the interactions of the users with Robota over 6 months - Ran from 10am until 6pm every day - 30 to 100 users per day (mean 43, std 13) - Interactions lasted between 2secunds and 5 minutes (mean 35 sec, std, 1.15min) - Wide range of nationalities (french, english, german, spanish, dutch, swedish) Type of interactions: ~45% (body words), ~20%( naturalistic), ~35% (insulting or abusive language)  Example of naturalistic trainings: « Drink », « Have an Apple », « Give me a kiss», « sing » Robots for Education

A.G. Billard, Autonomous Robots Class Spring 2007 Progressive exploration by a 2-year old child Robots for Therapy Kozima, H.; Nakagawa, C.; Yasuda, Y.; Interactive robots for communication-care: a case-study in autism therapy Robot and Human Interactive Communication, ROMAN IEEE International Workshop on Aug Page(s):

A.G. Billard, Autonomous Robots Class Spring 2007 Kozima, H.; Nakagawa, C.; Yasuda, Y.; Interactive robots for communication-care: a case-study in autism therapy Robot and Human Interactive Communication, ROMAN IEEE International Workshop on Aug Page(s): For more than a year and half (over 500 child-sessions), observed a group of children with autism, PDD, Asperger's syndrome, Down's syndrome, and other developmental disorders; Recorded live interactions between Keepon and the children from the perspective of Keepon Robots for Therapy Child seen from Keepon's eyes

A.G. Billard, Autonomous Robots Class Spring 2007 Kozima, H.; Nakagawa, C.; Yasuda, Y.; Interactive robots for communication-care: a case-study in autism therapy Robot and Human Interactive Communication, ROMAN IEEE International Workshop on Aug Page(s): For more than a year and half (over 500 child-sessions), observed a group of children with autism, PDD, Asperger's syndrome, Down's syndrome, and other developmental disorders; Recorded live interactions between Keepon and the children from the perspective of Keepon Robots for Therapy In the longitudinal observations, the children showed various actions in relation to Keepon. Sometimes they showed vivid facial expressions that even their parents had not seen before. They also showed prosocial actions like trying to feed Keepon, putting on a cap on its head, and kissing it.

A.G. Billard, Autonomous Robots Class Spring 2007 Case 1. Emergence of Dyadic Interaction The first case is a girl M with Kanner-type autism and mental retardation (MA 2:6 at CA 5:0; no apparent language). We describe the first 15 sessions, which lasted about 5 months (CA 3:9.4:1): M showed strong interest from Session 1 (hereafter S1), but did not get close to Keepon. Through S1 to S7, M avoided being looked straight at by Keepon (i.e., aversion to eye-contact); however M often looked into Keepon's profile. In S5, after observing another child R put a paper cylinder on Keepon's head, M dragged the nurse's arm, asking her to do the same thing to Keepon. When the nurse completed the task, M looked satisfied and left from Keepon. Through S5 to S10, the distance to Keepon gradually got shorter (to 40.50cm). In S11,M touched the head of Keepon with a xylophone stick, then M directly touched with her hand as if M had examined the texture and smell of Keepon. After this first touch, M started social interactions including eye-contact and vocalization (S12.), putting a cap on Keepon, asking her mother to do the same (S13.), and hugging and kissing Keepon (S14). We can see here the emergence of both spontaneous dyadic interactions, such as touching Keepon with a xylophone stick, and interpersonally triggered dyadic interaction, such as putting a paper cylinder on Keepon. Especially, the latter suggests that M was a good observer of others' behavior, while she seldom imitate when being instructed. Because R's action was mediated by Keepon and an object (e.g., the paper cylinder), which were being of N's interest, it would be relatively easy for N to emulate the same result. Interactive robots for communication-care: a case-study in autism therapy Kozima, H.; Nakagawa, C.; Yasuda, Y.; Robot and Human Interactive Communication, ROMAN IEEE International Workshop on Aug Page(s): Robot and Human Interactive Communication, ROMAN IEEE International Workshop on Robots for Therapy

A.G. Billard, Autonomous Robots Class Spring 2007 Case 2. Emergence of Triadic Interaction The second case is a 3-year-old girl N with Kannertype autism with mental retardation at a medium level (no language). We observed her interact with Keepon for totally 39 sessions, which lasted about 18 months: In S1, N gazed at Keepon for a long time. After observing another child W played with Keepon using a toy, N was encouraged to play with Keepon using the same toy, but N did not show interest in doing that. Through S2 to S14, N did not pay attention to Keepon, even when she was next to it. However, N often glanced at Keepon, when she noticed its sound, such as.Pong pong pong. In S15, after observing another child R put a cap on Keepon's head, N touched Keepon with her finger. In S16 (after 3-month blank from S15), N came close to Keepon and looked into its movement. In the snack time, N came to Keepon and poked its nose, to which Keepon responded by bobbing, and N showed surprise and smile. the mothers and nurses burst into laughter. During this play, N often made referential looks with smile to her mother and the nurse. From S17, N often sat in front of Keepon with her mother; sometimes she touched Keepon to derive some response. From S20, N started exploring Keepon's ability by walking around it to see if it could follow her. In the snack time of S33, N came to Keepon and started imitation play. its movements. when N performed one movement (bobbing, rocking, or bowing), soon Keepon mimicked; then N did another, and Keepon did the same. Through S33 to S39, N often played this imitation game. with Keepon, during which she often made referential looks to her mother and the nurse. Interactive robots for communication-care: a case-study in autism therapy Kozima, H.; Nakagawa, C.; Yasuda, Y.; Robot and Human Interactive Communication, ROMAN IEEE International Workshop on Aug Page(s): Robot and Human Interactive Communication, ROMAN IEEE International Workshop on Robots for Therapy

A.G. Billard, Autonomous Robots Class Spring 2007 Keepon elicited curiosity in children showing difficulty in interpersonal communication (especially, those with autism and PDD); its small size seem to elicit a sense of security. Some of the children extended their dyadic interaction with Keepon into triadic inter-personal interaction, where they tried to share with others the pleasure and surprise they found in Keepon. Each child showed a different style of interaction that changed over time; this provides information on the child’s personality and developmental profile, that is not thoroughly explained by a diagnostic label such as autism.. Interactive robots for communication-care: a case-study in autism therapy Kozima, H.; Nakagawa, C.; Yasuda, Y.; Robot and Human Interactive Communication, ROMAN IEEE International Workshop on Aug Page(s): Robot and Human Interactive Communication, ROMAN IEEE International Workshop on Robots for Therapy

A.G. Billard, Autonomous Robots Class Spring 2007 Promising results have been achieved in the use of toy-like robots for therapy with autistic children, sick children and with elderly people. More studies need to be conducted to determine the level of acceptability of these toys by a large population; Most of these toys are full-fledged prototypes ready to be commercialized. Next step is to move to bigger robots, for use with normal adults in our everyday life. SUMMARY