1. Towards Task-Oriented Computing for Pervasive Computing Environments Presenter: Chuong C. Vo Supervisors: Dr Torab Torabi & Dr Seng W. Loke 1 La Trobe University, 4/2010

2. Outlines Introduction to pervasive computing Usability problems with pervasive computing environments Approach Implementation Evaluation Research agenda Conclusion 2

3. Introduction The first era of computer (past, many-one) – Mainframe: central, heavy, and expensive The second (now, one-many) – Personal computers: personal, light, inexpensive The third (now & future, many-many) – Post-PC: pervasive (everywhere, every time), computers blend into environments. – Whole environments are computers. 3

4. Introduction (cont. 1) What is pervasive computing ( or ubiquitous computing )? – Seamlessly integrating computational elements into the fabric of everyday life…” [Weiser 1991], – Everyday objects and environments are aware of their surroundings & peers and behave smartly. The aims: – Support our activities, complement our skills, add to our pleasure, convenience, accomplishments [Norman 2007]. 4

5. Introduction (cont. 2) What is a pervasive computing environment? – E.g., smart spaces, smart environments, intelligent environments Pervasive city’s scenario – “It’s 7p.m., it’s raining, and you’re walking in Melbourne. You consult your phone and it suggests ‘Dinner?’, ‘Taxi?’, ‘Bus?’. Selecting ‘Dinner?’ will present restaurants you’re apt to like and even dishes that you may want…” 5 [cited from other work]

6. Introduction (cont. 3) Pervasive campus’ scenario – “You’re driving approaching La Trobe Uni. Campus, the LCD on your car suggests ‘Campus map?’, ‘Find a place?’, ‘Parking spot?’. Selecting ‘Parking spot’ will guide you to find a parking spot.” Pervasive personal office’s scenario – “You enter your office. The lighting, heating, and cooling levels are automatically adjusted based on you electronic profile. The coffeemaker works to give you a cup of hot white coffee.” 6 [cited from other work]

7. Outlines Introduction to pervasive computing Usability problems with pervasive computing environments Approach Implementation Evaluation Research agenda Conclusion 7

8. Usability Problems with Pervasive Environments Complexity of use: Variety of devices, UIs, remote controllers – Requires too many buttons and menus on UIs, exceeds capacity of UIs for users to operate them intuitively [Rich 2009]. – 1/2 of all reportedly malfunctioning consumer electronics products returned to stores are in full working order—customers just couldn’t figure out how to operate them. [Ouden 2006] 8 Slideshow controller TV controller A future pervasive computing environment Audio controller Blind controller TV Video-conference Display Temperature controller Projector DVD player Printer Computer Light controller Recorder Smart phone Door Coffeemaker

9. Usability problems (cont. 1) Invisibility & Overload of features – Technologies blend into environments [Pinto 2008]. – Frequently adding/removing devices and services to/from the environments. – Overload of features [Garlan et al. 2002]. One device  tens of features Different combinations of devices  Hundreds of features 9 MediaCup [Beigl et al. 2000]

10. Usability problems (cont. 2) Inconsistency of user interfaces – Brand identification, product differentiation [Rich 2009; Oliveira 2008] Inconsistency of task executions – Same tasks but different operations/procedures 10

11. Outlines Introduction to pervasive computing Usability problems with pervasive computing environments Approach Implementation Evaluation Research agenda Conclusion 11

12. Research hypothesis Our approach is based on task-driven computing [Wang et al. 2000] : – A task is a user’s goal or objective [Loke 2009]. – Users interact with/think of the computing in terms of tasks instead of applications/devices. – Users focus on the tasks at hand rather than on the means for achieving those tasks [Masuoka2003]. – Application function is modeled as tasks and subtasks. Our research investigates the FEASIBILITY, USABILITY, and EFFECTIVENESS of the task-oriented approach to the mentioned problems. 12

13. Approach: task-oriented framework ProblemProposed approach Complexity of useTask-based user interfaces Invisibility & overload of features Context-aware task recommendation Inconsistency of UIs & task executions Abstraction of task models 13

14. Overview of Task-oriented framework 14 Developer(s) /Designer(s) Task model Task models Task Repository Context-Aware Task Recommender Task Execution Engine Context Information Manager User(s) (1) Publishes (2a) Advertises models (2b) Provides context (3) Recommends tasks (4a) Executes tasks (4b) Provides context Focus of this talk

15. Context-aware task recommendation A combination of the following methods: – Location based – Pointing gesture based – Collaborative filtering using situation similarity 15 A set of all possible tasks Location-based recommendation Pointing gesture based recommendation Situation-based collaborative filtering

16. Location-based recommendation 16 Location = La Trobe University Campus Find a path Find a place Find a parking spot … Tasks

17. Location-based recommendation 17 Location = La Trobe University Campus Find a path Find a place Find a parking spot … Tasks Enroll a subject Find a room … Tasks Location = Building PS1

18. Location-based recommendation 18 Location = La Trobe University Campus Find a path Find a place Find a parking spot … Tasks Enroll a subject Find a room … Tasks Location = Building PS1 Location = Personal office PS1-219 Make coffee Dim lights Watch TV … Tasks

19. Location-based recommendation 19 Location = La Trobe University Campus Find a path Find a place Find a parking spot … Tasks Enroll a subject Find a room … Tasks Location = Building PS1 Location = Personal office PS1-219 Make coffee Dim lights Watch TV … Tasks Location = TV’s zone Make coffee Dim lights Watch TV … Tasks

20. Pointing gesture-based recommendation 20 Location = the Agora Find a place Meet friend Coffee … Tasks Theatre Coffee shop …

21. Pointing gesture-based recommendation 21 Location = the Agora Find a place Meet friend Coffee … Tasks Theatre Coffee shop … Pointing at = the Theatre What is on? Special offer? Ticket booking … Tasks

22. Pointing gesture-based recommendation 22 Location = the Agora Find a place Meet friend Coffee … Tasks Theatre Coffee shop … Pointing at = the Coffee shop What is on? Special offer? Ticket booking … Tasks Coffee Food Special offer? … Tasks

23. Pointing gesture-based recommendation 23 Location = Personal office TV Air-conditioner Make coffee Dim lights Watch TV … Tasks

24. Pointing gesture-based recommendation 24 Location = Personal office Make coffee Dim lights Watch TV … Tasks Pointing at = the Air-conditioner TV Air-conditioner Temperature up Temperature down Set fan speed … Tasks

25. Situation-based collaborative filtering – Assumption: “People tend to accomplish similar tasks in similar situations”. – The tasks previously accomplished by similar users in similar situations are recommended. Steps: 25 Find similar users Find similar situations Rate tasks done by the similar users in the similar situations

26. Situation-based collaborative filtering Situation similarity – A situation is defined as a vector of context attributes: s = (c 1, c 2,…, c n ). E.g., s = (role=‘Student’, time=‘Monday’, location=‘Library’). – Similarity between two situations, s and s’: is the number of common tasks and is the number of all tasks typically accomplished in these situations. 26

27. Situation-based collaborative filtering User similarity – Similarity between two users u, u’ is the sum of the similarity of their attributes such as age, gender, role, and occupation: Where denotes the significance weight assigned to attribute a i ; sim i (a i,a’ i ) is the per-attribute similarity between two values a i and a’ i for the attribute i. 27

28. Outlines Introduction to pervasive computing Usability problems with pervasive computing environments Approach Implementation Evaluation Research agenda Conclusion 28

29. Implementation Technologies for estimating locations 29

30. Implementation 30 Location-based task recommendation Location = University campusLocation = Personal office

31. Television Implementation 31 Indoor pointing: uses Cricket system Outdoor pointing: uses iPhones with compass & GPS built-in Air-conditioner Cricket Listener Cricket Beacon pointing at TV pointing at Air-conditioner tasking

32. Implementation 32 Executing tasks Power line Task Execution Engine Wirelessly execute tasks We’ve implemented some simple device- based tasks using X10 technology Kettle Light

33. Outlines Introduction to pervasive computing Usability problems with pervasive environments Approach Implementation Evaluation Research agenda Conclusion 33

34. How to evaluate our approach Metrics and methods: – Feasibility Implementing a prototype and evaluate performance – Usability Simulation & User evaluation (questionnaires) – Effectiveness Observe what tasks being selected by users and compare them with the recommended tasks in a number of situations. Record recall and precision of task recommendation. 34

35. Work to be done next Implementing the Task Execution Engine to support more complex tasks, Evaluating the framework, the techniques used, and the design decisions made. Extend task models to incorporate context information into task execution. 35

36. Future work Extend the task model description standard, Build a graphical tool for authoring task models, Develop mechanisms for effectively publishing and retrieving task models, Indexing, matching, searching, composing, recognizing task models Address conflicts of task executions in multi- user environments. 36

37. Publications C. Vo, T. Torabi, and S. Loke. Towards context- aware task recommendation. In ICPCA-09, Taiwan, 2009. C. Vo, T. Torabi, and S. Loke. T ASK R EC : A Task- Based User Interface for Smart Spaces. Submitted for review to UBICOMP 2010. C. Vo, T. Torabi, and S. Loke. Towards a Task- Oriented Framework for Smart Spaces. Submitted for review to SOOW 2010. 37

38. Conclusion Defined usability problems with pervasive computing systems: – Complex of use – Invisibility & overload of features – Inconsistency of UI Presented the task-oriented approach, its implementation, and evaluation methodology, Outlined the research agenda. 38

39. Outlines Introduction to pervasive computing Usability problems with pervasive environments Approach Implementation Evaluation Research agenda Conclusion 39

40. Questions? Thank you! 40

41. Related work Situation-aware application recommendation [Cheng et al. 2008] – They recommend applications <> We recommend tasks (multi-apps) – They use pure situation similarity <> We use task based similarity Homebird system [Rantapuska et al. 2008] – It recommends tasks based on features of devices discovered – However, because this approach does not consider user situation, it can recommend feasible tasks which may be not relevant. InterPlay [Messer et al. 2006] – For device integration and task orchestration in a networked home. – It asks user to express their intended tasks and assumes that the users have knowledge about feasible tasks. – In contrast, our approach can recommend relevant, feasible tasks without these requirements. Context-dependent task discovery [Ni et al. 2006] – Discovering active tasks by matching current context with required context of tasks. – This can discover feasible tasks but potentially irrelevant tasks. Task retrieval [Fukazawa et al. 2005] – Ask user to specify target names (e.g., cafe shop, theatre) for retrieving tasks which are associated with these names. – Our system has integrated this knowledge into place/devices-related task repository. 41

42. References M. Weiser, “The computer for the 21st century,” Sci. American, 3(265), pp. 94–104, 1991. D. A. Norman, The Design of Future Things. Basic Books, 2007. Z. Wang & D. Garlan, “Task-driven computing,” School of Computer Science, Carnegie Mellon University, Tech. Rep., 2000. D. Garlan et al. “Project Aura: toward distraction-free pervasive computing,” Pervasive Computing, IEEE, 1(2), pp. 22–31, 2002. R. Masuoka et al. “Task computing – the semantic web meets pervasive computing,” The SemanticWeb, pp. 866–881, 2003. D. Magnusson & B. Ekehammar, “Similar situations–similar behaviors? a study of the intraindividual congruence between situation perception and situation reactions,” J. of Research in Personality, 12, pp. 41–48, 1978. A. K. Dey, “Understanding and using context,” Per. and Ubi. Computing, 5(1), pp. 4–7, 2001. D. Cheng et al. “Mobile situation-aware task recommendation application,” in The 2 nd Int. Conf. on Next Generation Mobile App., Services, and Tech., 2008. A.Messer et al. “InterPlay: A middleware for seamless device integration and task orchestration in a networked home,” in PERCOM’06. 2006, pp. 296–307. H. Ni et al. “Context-dependent task computing in pervasive environment,” Ubi. Comp. Sys., pp. 119–128, 2006. Y. Fukazawa et al. “A framework for task retrieval in task-oriented service navigation system,” in OTM Workshops 2005, pp. 876–885. O. Rantapuska and M. Lahteenmaki, “Homebird–task-based user experience for home networks and smart spaces,” in PERMID 2008, 2008. 42