1. REU 2007 presents… AVARI Animated Virtual Agent Retrieving Information

2. Your Name2 2 Vision, Communication Shell, & Conversational Flow Lauren Cairco, primary Toni Bloodworth, secondary

3. Your Name3 3 Background & Motivation [Thorrison, 1998] [Cassell, 2001] [Babu, 2005] [Simmons, 2002]

4. Your Name4 4 Who is AVARI?

5. Your Name5 Who is AVARI? Animated Virtual Agent Retrieving Information Answers questions about members of the computer science faculty of UNC Charlotte 5

6. Your Name6 6 General Architecture Animated Agent (Haptek) Information Display (HTML/Javascript) Vision (Matlab) Voice Recognition (SALT) Database (MySQL) Communication Shell (Javascript) Question Resolution Algorithm (PHP)

7. Your Name7 7 My Contributions Vision Communication Shell Conversational Flow

8. Your Name8 8 Vision Background subtraction using Matlab

9. Your Name9 9 Vision Camera looks at the floor to see if anyone is standing in front of Avari Begins conversation when she sees someone Ends conversation when person leaves

10. Your Name10 Vision

11. Your Name11 Communication Shell Writing code to hold all the pieces together –PHP (PHP Hypertext Processor) –SALT (Speech Application Language Tags) –Haptek (for virtual human) –Matlab (for vision) Communicating with group members about needs from other pieces Version control

12. Your Name12 Conversational Flow Greeting Get Professor Name Get Category Get Question Goodbye Say Answer

13. Your Name13 Greeting Get Professor Name Get Category Get Question Goodbye Say Answer Conversational Flow Get Professor Name Get Category Keep professor? Keep talking? YesNo YesNo Prompt for professor User says professor name Keep category? YesNo Prompt for category User says category

14. Your Name14 Conversational Flow Avari’s phrases that prompt users are randomly selected from text files An additional screen displays information that is relevant to the conversation Avari gives users clearer cues if she does not recognize their speech multiple times –Keywords on the other screen –Example questions

15. Your Name15 Sample Conversation

16. Your Name16 Challenges in Design Predicting what people will say to Avari Deciding how to best direct conversation while keeping it natural

17. Your Name17 Challenges in Implementation Communication problems –Server side and client side languages –Integrating vision software Security restrictions in Internet Explorer Timing and efficiency considerations

18. Your Name18 Skills I Learned PHP, SALT, MySQL, Matlab, Java, Javascript, Haptek How to work with other people’s code Debugging without a compiler

19. Your Name19 Vicky Fowler Animated Agent Morris Leblanc Speech Recognition Louis Fletcher Database Toni Bloodworth Question Resolution Algorithm

20. Your Name20