1. Dr. Fuhua Lin International Graduate Research Workshop 2013 on Adaptivity and Personalization in Informatics, March 23‐24, 2013 Edmonton, Alberta, Canada

2. Outline Introduction Related Work The Roles of Agents Architectural Design QuizMASter Conclusions and Future work

3. Goal of this Tutorial Discuss the basic concepts on constructing virtual worlds with multiagent systems (MAS).

4. Virtual Worlds Virtual Worlds are 3D graphical environments that provides more engaging and more immersive user experience can provide more dimensions than physical environments more social nuanced ways (in ways which are able to show slight difference that may be difficult to notice but are fairly important ) for people.

5. Athabasca University’s Academic Research Centre in Second Life AU's new ARC building was exactly replicated on the AU island to assist with furniture placement and other occupancy decisions. A video of this building is accessible here: http://www.youtube.com/watch?v=pganB4_F7AQ There are avatar only in this demo! The avatars are 3D characters controlled by the participants.

6. Multiagent Systems MAS --- encompasses distributed problem-solving applications, such as network management, which do not typically involve Virtual Worlds has a focus on inter-agent Communication. Coordination and Negotiation. COMP667 (Multiagent Systems) of Athabasca University

7. Group Decision Making There are many settings where a potentially large number of agents, each with its own goals and objectives, collectively interact so as to produce a solution to some problem. A solution that is produced under these circumstances often reflects the “tug-of-war” that led to it, with each agent trying to pull the solution in a direction that is favorable to it.

8. Game Theory The field of game theory provides a natural framework in which to talk about what happens in such situations, when a collection of agents interacts strategically --- in other words, with each trying to optimize an individual objective function.

9. Avatars NPCs Software agent 1 Software agent 2 Software agent 3 Software agent 4 Software agent 5 Software agent 6 Software agent 7 Software agent 8

10. Autonomous Agents Russell and Norvig,

11. Autonomous Agents in Virtual Worlds Using virtual worlds as a technique for exploring agent behavior and agent believability, or Using agents as a way of extending virtual worlds into new application areas, including Synthetic agents: intelligent actors (Hayes-Roth et al. 1996), virtual actors, virtual humans, (e.g. NPCs) Avatars (which are physical representation of human users) In a 3D multi-user Web environments.

12. AGENTS Exist in an environment which they need to perceive Autonomously react to changes in that environment Proactively act to achieve goals Interact with other agents Behavior can be emergent (often coded in a declarative language) Belief-Desire-Intention (BDI) models a subset of the human mind

13. Why Autonomous? The more autonomous, the more convincing to the user and sustains the feeling of presence in a virtual world!

14. Behavior Modeling: Agents’ Level of Autonomy (LOA) Degree of autonomy of virtual environments depends on the autonomy of their components, three level of autonomy (LOA) (Thalmann, 2000): Guided: user-guided, like avatar. Programmed: could use omniscient approach, but inefficient. Autonomous: Have perception limitation Prediction about the world are always fallible The potential for reuse of agents in different virtual worlds The ability to distribute individual agents over separate processors

15. Is Autonomy useful and appropriate for agents in virtual worlds? Omniscient agent management soon runs into combinatorial problems when it must track of what each agent is supported to know and perceive.

16. NPCs Realistic non-player characters (NPCs) are essential to making virtual environments more real for players. This is true in video games where more believable NPCs support the story narrative of a game, making them more immersive, more convincing. It is also true in other areas where virtual worlds are used such as education, increasing the effectiveness of those environments.

17. PROBLEM Non-player Character (NPC) behaviors in 3D virtual worlds are not sophisticated enough Impacts believability and ability to create complex scenarios Domains Video games Education/Training virtual environments

18. SOLUTION Improved Artificial Intelligence driving NPC behavior Machine Learning Natural Language Processing Machine Perception (watching the user) Multi-agent Systems (MAS)

19. POSSIBLE APPROACHES Two approaches to combining an MAS with a game engine A fully custom integrated solution A modular solution, integrating existing components

20. Gamebots A fully custom integrated solution

21. Virtual Singapora

22. Problem in using MAS in Virtual Worlds Game AI is usually closely coupled with other parts of the game code which makes it hard to reuse or replace. Requires a large amount of investment in time and resources and a high level of expertise in Agent Oriented Software Engineering (AOSE).

23. GOAL Create an integrated framework to simplify the process of creating agents to control NPCs in Open Wonderland Animation Movement Environmental percepts Synchronization

24. PAST RESEARCH Gamebots/Pogamut (2002 - “ GameBots: A Flexible Test Bed for Multiagent Team Research ” ) Commercial server (Unreal Engine) Open source client (using Unreal scripting language as API) Other Projects Not Java-based (C, C#) or not open source OpenSim (C#) supports Second Life protocol

25. Agents’ Roles Research that has been done with virtual agents and multi- agent systems can be leveraged to create more realistic NPCs and purposeful communication channels among agents for applications like game-style educational activities. Agent- controlled- NPC Human User’s Avatar

26. Our Approach Controlling NPCs with intelligent agents through the creation of an interface between a multi-agent system to a virtual world engine.

27. Open Wonderland Open Wonderland is a toolkit for building 3D virtual worlds (Kaplan, J., & Yankelovich, 2011). The architecture of the system, based entirely on open standards, is highly modular and designed with a focus on extensibility. Kaplan, J., & Yankelovich, N.: Open Wonderland: An Extensible Virtual World Architecture. ;IEEE Internet Computing(2011), 38-45

28. Underlying Technologies Jason A platform for the development of multi-agent systems Java-based open-source MAS used for creating BDI agents that are based on the model of belief, desires, intentions. A Java-based interpreter for an extended version of AgentSpeak. The core code is easily extendible making it easy to add customizations at the agent and environment level. AgentSpeak, a Prolog derivative that is well suited to programming in AI, is the language used to control agents. AgentSpeak is only language that will be supported, at least initially, when it comes to agent scripts run to control NPCs in Open Wonderland. Declarative style --- offers a different way to script NPC behavior than the more functional methods already available in Open Wonderland (like JavaScript).

29. Underlying Technologies JADE Container model Agent Management System (AMS) Directory Facilitator (DF) to facilitate the management of agents. CArtAGo – (Common ARTifact infrastructure for AGents Open environments) Agents & Artifacts (A&A) model, using workspaces, agents, and artifacts (exposed resources).

30. MEMORY

31. Perception

32. A Client with TextChat Echo Agent

33. Proximity Detection /* Initial goals */ !listen. @msgHandler[atomic] +proximity(FromUser,Enter,Distance) : proximity(_,_,_)[source(percept)] <-.print(FromUser); text_chat_send_message(FromUser, "Proximity Agent", ""). @listenHandler[atomic] +!listen : true <- proximity_listen; text_chat_send_message("listening to proximity messages", "JADE Controller", ""); cheer.

34. MOVEMENT AND ANIMATION

36. Interactions among Agents

37. Quiz Games in Classrooms In classrooms, teachers usually use quiz games to create some interesting activities. The purposes of Quiz Games for the classes Good for reviewing and reinforcing previously taught material Good for warming up or ending lesson on a high A quiz encapsulates the basic unit of conversation

38. Quiz Master Quiz master A TV game show, where a small group of contestants compete by answering questions presented by the game show host. Host -  Pedagogical agent

39. Purpose of the Project Build QuizMASter a virtual world based educational application that mimics a quiz game in classrooms. To build engaging, affectionate, and effective pedagogical agents which is the virtual host in QuizMASter. How?

41. Transformed Social Interaction – TSI Theory (Bailenson et al. 2008) to describe the transformation of interaction in mediated communication environments. Three principles of TSI

42. Principle 1: Self-Representation There is evidence in support of the advantage of self- representation. For example, Bailenson, et al. found that the morphing of faces of political candidates with potential voters will increase the affective bonding in low information context (Bailenson et al., 2006)

43. Principle 1: Self-Representation (Con’t) the dramatic and subtle changes to the Appearances Behaviors of the avatars chosen by the users, such that emotional bonding can be established. user

44. Implementation of the TSI dimensions Self-representation – morphing of real faces into the avatars = User 1: Oscar’s face Generic pedagogical agent: Stephen User 2: Steve’s face 40:60 In shape only Transformed face of the pedagogical agent that Oscar will see in the virtual world Transformed face of the pedagogical agent that Steve will see in the virtual world eVolver (http/l//www.evolver.com)

45. Principle 2: Sensory-Abilities: These transformations empower avatars to complement human perceptual abilities. Agent-enhanced host avatar Agent technology’s roles: the sensory ability to process the behaviors of contestants: eye gazing or moving data emotional status Detect students’ personal taint and preferences to provide personal greeting, etc. Learn about knowledge to make decision Provide advice/reminders to contestants. Express emotions to the contestants

46. Implementation of Sensory abilities Applied by recording and observing the student avatar behaviors, especially his/her gaze direction. After all contestants enter the studio, the Host Agent will greet each and every student, addressing his/her name with eye-gaze directed to that student. User monitor --- tracking of viewing angles of participants Contestant Name: Lin Data: Lin's Look-Around behaviour during every question session Date: Fri Sep 23 22:01:48 MDT 2011 Question Number 1 Session -->Lin Did Not Looked Around. Question Number 2 Session -->Lin Did Not Looked Around. Question Number 3 Session -->Lin Did Not Looked Around. Question Number 5 Session -->Lin Did Not Looked Around.

47. Principle 3: Situational Context Transformations Altering the spatial or temporal structure of a conversation. For example, the communication between agents and students can be optimally configured in terms of the geographical setup of a conference room. For example, a class of 20 students can sit directly in front of the virtual instructor, and perceive the rest of the students as sitting farther away.

48. Principle 3: Situational Context (Con’t) Altering the flow of rendered time in the communication session, users can implement strategic uses of rewind and fast forward during a real-time interaction in an attempt to increase comprehension and efficiency. E.g. the point for providing hints …

49. Scenario Participants Contestants The host The Audience Process of a QuizMASter session The host greets the contestants The audience will be cheering for and looking at all the contestants simultaneously. Questions are displayed. Contestants will answer the questions. Scores will be kept. Another video showing the introduction of the host and the audience responses.

50. Demo http://www.youtube.com/watch?v=P0Gum- RaMfo&feature=player_embedded http://www.youtube.com/watch?v=P0Gum- RaMfo&feature=player_embedded http://www.youtube.com/watch?v=jPQLnZHewFg http://www.youtube.com/watch?v=r-d4FSqpoV0