Design criteria, techniques and case studies for creating and evaluating interactive experiences Arjan Egges Anton Eliëns Jonathan Gratch Katherine Isbister Stacy Marsella Ana Paiva Thomas Rist Paul ten Hagen

Starting Point ECA research looks to human-human interaction –As a source of theory and data –As the ideal Ecological validity ≠ Functional validity –Super elicitors –Artificial Intelligence evolved away from human cognition Sense that non-reality can be functional for ECA –“Dutch light” –Exaggeration If we leave reality, what are our signposts? –What fields and paradigms can inform ECA design – What criteria can judge success

Assumptions Interaction is central ECA’s primary function is to promote effective HCI Directly with the ECA Indirectly with a system Interaction can be analyzed as a graph structure –Specify what a user can do, not how they do it Do not represent user beliefs, motivations, etc. –Nodes are states of the system/ECA –Arcs are user “moves”: dialogue, mouse clicks, etc. Success defined as some policy over graph –User must reach some end state subject to constraints –User should follow certain trajectories

Example: MRE Who is hurt? What happened? Get Ambulance Caveats –Graph an analytic tool, not necessarily truth –Alternative views of what constitute nodes and arcs

What is the (implicit) graph and/or space? –What are the rules of causality and moves in the space –How do you enforce closure (prevent departure from space) What is the desired path through this space? –What are acceptable end states –What are the criteria/constraints on a desirable path How is user guided toward the desired path? –By the ECA –By other cues What measurable criteria can help with answers Interaction design “mindset”

Guidance Key question is how to provide guidance Emphasize role of “affordances” –Coined by perceptual psychologist James Gibson –Applied to many domains (HCI, product design, …) An affordance is a cue to act: –A chair affords ("is for") support, and, therefore, affords sitting." (Don Norman) –An affordance is a relationship between an object in the world and the intentions, perceptions, and capabilities of a person (John Seeley Brown) –Encapsulates physical and mental

Guidance How do affordances that the ECA provides interact with the topology of the graph and desired policy –Behaviors that suggests actions outside the graph are BAD

Assessment Criteria Closure and Reachable –Test both in absolute sense -> mechanically –Can desirable end states be reached –Are desirable paths “connected” Psychological variables –Affordance is user interpretation of cues –Can assess in terms of Impact on intrinsic motivation Sense of control Sense of challenge Sense of novelty

Case Studies Design more like an art than a science Performed a number of case studies –Curse of Monkey Island –MRE –Fantesea –Rudy

What is the (implicit) graph and/or space –Click on objects. You can do an exhaustive search –Talk to characters: small conversation graph –Very limited options and strong affordances. What is the desirable path through this space –get user to some node (End state) : well defined exit conditions specified up front –get some interaction (Be entertained) –Explicit local goals that you get rewarded for achieving – creates a sense of control –Prevent boredom –(Note that there is some balance between effort and reward – more effort leads to greater reward) What techniques are used to cue user toward desirable path –Back story establishes user persona, helps constrain (telegraph) what you can do: a comic hero. Also giving you your goals. Save girlfriend. Do in the bad pirate. –PT 1: the demise of the bad pirate: giving your goal –Multiple choice way of talking to character –Character giving very local goal description (I could open the door) –User trained to interact: given an initial encounter that frames it explicitly. –Mouse gives visual feedback on what you can click on (turns red) –The initial scene forces you to learn all the skills you need before you solve the puzzle. In some sense this introduces parallelism/nonlinearith. Required to learn skill A, B,C to exit room. How did/might the designers assess success –play testing Curse of Monkey Island

What is the (implicit) graph and/or space –Dialog moves – info about states, what should we do, give orders –Task model (small number of possible states) What is the desirable path through this space –There is a “good” outcome and a “bad” outcome –What is the goal: immersion (perceptual cues and info cues from being immersed) Get user stressed. Force them to make an unpleasant choice and discuss it afterwards. Also memory function to help recall –What are the criteria/constraints on a desirable path There is this dream of open ended interaction with and ECA (how do we move monkey island more towards MRE) What is the path Are there local rewards? Getting the sgt understand you is similar to the frustration of getting off a level. Is the communication problem “part of the fun?” (as exploited in certain games) or does this run contrary to the immersive goal? What techniques are used to cue user toward desirable path I can’t understand you – pops out of conversation (narrative?) Constraints of the role Dialogue failure feedback – should give differential feedback based on couldn’t hear, couldn’t parse, outside domain. Clarification dialog, initiative I: would expect this is not so much about dialogue but getting the person emotionally involved in the pressure and dilemma. More important is body language, emotional cues. How did/might the designers assess success –Spend time w/ users identifying boundary conditions MRE: (how does the user know what to say)

MRE (continued) The sgt turning to the student is cue that they can speak –This gives you a problem – to turn towards means you have to turn away. One interpretation of turning away is he is trying to drawing your attention to the boy. –Turning away may lower sense of control. Conveys that SGT does not respect the user which should lower control. –Also a staging error. Sgt too close to the camera. The staging forces these odd cues. The turning to the medic and restating the question give the sense that you can’t talk to the medic directly. Whereas if he just turned, you might get the sense that you could Relationship of graph moves Trying to secure lz first-> sgt tell you not to do that but some cases we messed up. The model enforces certain dependencies that the cadet’s don’t have (cant send 1 sqd forward and then secure area – their training doesn’t lead them to believe that. Questions – as scene deteriorates, do the characters convey this through their behavior? Do they get more nervous – yes but this is not manifest. What is the consequence of the character seeming wooden not seeming wooden takes control from the user (it is autonomous). And thus there is more reward when he does have control. If you want to keep motivated, need a certain challenge

What is the (implicit) graph and/or space –Interact with toy. –Battle actions. –Structuring of goals to improve character What is the desirable path through this space –What is the goal: learn about emotions(?) What techniques are used to cue user toward desirable path Told some back story. 4 clans that have certain powers. You will have a duel and they way you fight is to influence the emotions of your character –Clans have different appearences and gestures (these make it easier to distinguish/recognize clans but are not intended to influence the interaction) Camera moves focus attention on gestures which are key for understanding rules of game sound Added sign to say “your turn” and character’s that are casting have circles around themselves MRE the central technical hurdle is what to say. Here the problem is what to do. –Identify What is artificial and lacking and focuse the cues/affordiances on overcoming this How do you get cues of what to do How did/might the designers assess success –Spend time w/ users identifying boundary conditions –Can we confirm if we are getting the level of immersion we want –Can easily play the thing at a meta-level: what are the rules/limitations, this runs contrary to the goal of immersing them. FantasyA:

What is the (implicit) graph and/or space –Chatterbot. Text input, and some response. Presentation strategy. What is the desirable path through this space –What is the goal: provide useful information –What are the criteria/constraints on a desirable path What techniques are used to cue user toward desirable path Cues in source code Entertain the user to get them to interact with it more? Motivation to start interacting? –Ellm Brine (Edinburgh) studied how people dress influences people’s interaction. A scruffy agent would not be trusted as much in a banking situation Dressed like a businessman, has a graph. Unclear how ECA influence interaction How did/might the designers assess success –Spend time w/ users identifying boundary conditions –Can we confirm if we are getting the level of immersion we want Rudy stock market agent

Future What’s in a graph More case studies of the framework Growth of terminology Relation to parallel efforts What terminology is used in these communities How do their formalisms map onto –Game design –eric zimmerman, katie sahlens, richard rouse, gamasutra.com, ludology.org –Education environment design –Dialogue community May not be good measures for the ‘success of the conversation’ which is essentially what we are trying to answer