Download presentation
Presentation is loading. Please wait.
Published byAnthony Dunklin Modified over 9 years ago
1
Complementing Embodied Conversational Agents by Implementing Compliments Measuring the Effect of Compliments in Embodied Conversational Agents Group 23 Marieke Agterbos | Tim van Bremen | Carel Jansen | Lisa Oud | Justin Post
2
Research Question To what extent do compliments have an impact on the user experience of embodied conversational agents?
3
Questions 1.Are the ECA’s compliments recognized as compliments? (Part I) 2.Experience with vs. without compliments (Part II) A.Perceived liking of the agent B.Perceived level of comfort C.Perceived information quality
4
Background theory Making machines more human (Picard, 2000) Minimize gap between real life contact and digital contact Lifelike ECA (Louwerse et al., 2009) Is sensitive to moods and sentiments Politeness and variation in language (Behnam & A. Niloufar, 2011) Add value and believability Function of Compliments (Manes en Wolfson, 1981) Goodwill and solidarity Three types of personal compliments (Jucker, 2009) Implicit, explicit or indirect Behavioral realism might be more important than photorealism (Groom, 2009) Generating positive response More productive conversation
5
Pre-test Are the ECA’s compliments recognized as compliments? Participants read a transcript of an ECA conversation Questionnaire on different parts of the conversation: 1. Bot: ‘Hello, how are you?’ 2. User: ‘Great, and you?’ 3. Bot: ‘Good, so am I.’ 4. User: ‘Good.’ 5. Bot: ‘What was the name of the lead guitar player of the band called “A band of Gypsies?”’ 6. User: ‘Jimi Hendrix.’ 7. Bot: ‘That is correct.’ 8. Bot: ‘Someone told me you are very bright.’
6
Pre-test results ‘Yes, this is a significant difference (p<0.01)’ 84 participants (53, 31)
7
Pre-test results
8
Experiment Design Measured variables Liking Comfort Information Quality
9
Experiment 53 Participants Conversation with bot Quiz setting Questionnaire Perceived liking Perceived level of comfort Information quality “How many strings does a guitar have?” “Who is the prime minister of the Netherlands?”
10
Results
11
Effects
12
Results H1. Contextual compliments have a positive effect on the user experience of an ECA. H1-a. perceived liking of the agent. H1-b. perceived level of comfort. H1-c. perceived quality of information. H2. Non-Contextual compliments have a positive effect on the user experience of an ECA. H2-a. perceived liking of the agent. H2-b. perceived level of comfort. H2-c. perceived quality of information. H3. Contextual compliments have a more positive effect than non-contextual compliments. H3-a. perceived liking of the agent. H3-b. perceived quality of information. H3-c. perceived level of comfort.
13
Results H1. Contextual compliments have a positive effect on non-contextual compliments on the user experience of an ECA. H2. Non-Contextual compliments have a positive effect on the user experience of an ECA. H3. Contextual compliments have a more positive effect than non- contextual compliments. H4. Compliments have a positive effect on the user experience of an ECA. H4-a. perceived liking of the agent. H4-b. perceived quality of information. H4-c. perceived level of comfort.
14
Conclusion Compliments have a positive effect on the user experience of an ECA. However, not on likingness. The type of compliment matters Non-contextual > contextual
15
Discussion Pros: Pretest, kind of compliments accounted for. Pretest in line with experiment outcome. All experiments under supervision of a test leader. Improvements: Sample too small Language, not native language of all subjects Variable mount of compliments given Compliments when correctly answered Definition of comfort, persuasiveness, likeliness might differ individually
16
Questions?
17
Measuring - Liking ‘I would like to be friends with her’ ‘If I was upset, she would be able to cheer me up.’ ‘She is a bad listener’ Rated on Likert scale 1-7
18
Measuring - Level of comfort During the conversation, I felt: Comfortable Involved Open Relaxed Warm Rated on Likert scale 1-7
19
Measuring - Information quality The extent to which the information was: Correct Helpful Persuasive Insightful Relevant Rated on Likert scale 1-7
20
Context Controlled environment Participants from same population
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.