1 PLAN RECOGNITION & USER INTERFACES Sony Jacob March 4 th, 2005

2 AGENDA Motivation – Examples Introduction – Collaboration Plan Recognition vs. Traditional Systems Plan Recognition System “Steve” Conclusions Discussion

3 EXAMPLE: TRADITIONAL SYSTEM Microsoft Interactive help system Provides “jump-in” help instead of relevant on-going collaborative help Unable to comprehend overall goals and does not use task model Most users are unable to use this feature successfully

4 EXAMPLE: PLAN RECOGNITION SYSTEM Charles Rich Candy Sidner Neal Lesh Andrew Garland Shane Booth Markus Chimani 2004 Mistubishi Electric Research Laboratory

5 EXAMPLE: PLAN RECOGNITION SYSTEM Charles Rich Candy Sidner Neal Lesh Andrew Garland Shane Booth Markus Chimani 2004 Mistubishi Electric Research Laboratory

6 MOTIVATION Minimize amount of initiative required from user Create simple and consistent interfaces Guide user without limiting capability

7 COLLABORATION DIAGRAM SHARED ARTIFACT (GUI) GOALS COMMUNICATION PRIMIVITIVE ACTIONS PRIMIVITIVE ACTIONS Model for Plan Recognition in user interfaces

8 COLLABORATION FRAMEWORK Defined as an Interaction between “Agent” and User – Mutual Goals – Agent and user can perform actions – Agent uses Plan Tree Hierarchal partially ordered representation of actions to achieve goals Methods of interaction – Discussion between agent and user – User conveys intentions through actions – Agent solicits clarification

9 EXAMPLE: PLAN TREE Charles Rich Neal Lesh Andrew Garland 2002 Mistubishi Electric Research Laboratory

10 EXAMPLE: PLAN RECOGNIZER Charles Rich Candy Sidner Neal Lesh 1998 Mistubishi Electric Research Laboratory

11 RESPONSES Possible actions for an agent – Move user to next step of task (goal oriented) – Confirm completion of a goal – Allow user initiative – Focus user to current goal – Explain steps needed for a task – Discover and report incorrect actions

12 RESPONSES CONTINUED Traditional responses – Application dependent If(user pressed button A) Call function A Collaborative responses – Application independent If(user completed a step in current task) Go to next step of task

13 INTERACTION Traditional system – Limited range 1. Tutoring systems – Agent has majority of plan knowledge and initiative 2. Help system – User has majority of plan knowledge and initiative – Turn based interaction User performs action and agent responds

14 INTERACTION CONTINUED Collaborative System – Broad range System can shift incrementally within this range – Examples mentioned in Traditional systems represent extremes of this range Mode depends on current task – Non-turn based interaction User may perform 0 or more actions followed by 1 or more communications Agent may perform 0 or more actions followed by 1 or more communications

15 COMMUNICATION Traditional Communication – User initiates system actions through “commands” – Requires user to have knowledge of command Collaborative Communication – On-going Discourse between agent and user Define goals and how to achieve them Discuss task being performed – Requires user to have common goal with agent

16 HELP SYSTEMS Traditional help systems – Wizards, Tool-tips, Help Assistants – Attempt to compensate for lack of user knowledge – Requires separate interaction by user Collaborative help system – Integrated as part of the interface – User knowledge level does not affect level of help system interaction

17 EXAMPLE: WIZARDS Wizards – Provide a guided interaction for user – Partially follows collaborative paradigm Lacks versatility to allow user to take initiative Goals cannot be adjusted

18 ADVANTAGES OF DIAMOND HELP Provides consistent interaction paradigm – Different applications of Diamond help will be familiar to user – Appearance and operation remains the same Used for appliances and control systems Possible expansion allows for speech- enabled interaction – Agent speaks interaction and performs speech recognition for user

19 PLAN RECOGNITION SYSTEM EXAMPLE: STEVE Training agent – Steve (Soar Training Expert for Virtual Environments)Soar Virtual reality tutoring system, agent embodiment Uses plan recognition to guide user Actions taken by user are interpreted and compared to plan tree Steve orients user towards goal based on plan tree Advises user when a deviation is made from the plan tree or when help is needed for the next step Demo –

20 CONCLUSIONS Effective Collaboration – Abstract representation of situation Key to reuse and modularity of components – Well-designed task model Hierarchy must model tasks which complete a goal or sub-goal – Focus must be maintained If goal is modified, focus “stack” must be adjusted

21 CONCLUSIONS CONTINUED Complexity and scalability – Must be able to create abstract representations for various tasks – Depends on modularity and reusability of components – For complex interactions, must allow more direct user actions – Require Sub-goals for top goals

22 DISCUSSION Questions? How do we program a collaborative system? What are the drawbacks of creating this system? How does the agent realize a plan for a complex system? How would I implement this in my workplace environment?

23 PROGRAMMING TECHNIQUE Object Oriented Plug-ins – Use separate components to construct desired interface Composable and reusable components Abstract class definitions for dialogs Low-level functionality is controlled and monitored by high-level plan recognition system

24 References Note: Web addresses only