1. A Toolkit for Managing User Attention in Peripheral Displays Tara Matthews, Anind K. Dey*٭, Jennifer Mankoff٭, Scott Carter, Tye Rattenbury EECS Department, UC Berkeley *Intel-Berkeley Research ٭currently of HCII, Carnegie Mellon

2. 10/27/2004 Peripheral Display Toolkit2 Problematic Scenario Bus schedule –Requires a task switch to use –Doesn’t afford continuous monitoring RTE LAWR GRIZ EUCL CNTR NUM HALL SR. GRIZ SHAT 65 605a 616a 630a 65 635a 646a 700a 65 703a 716a 730a 65 735a 746a 800a 65 803a 816a 830a 65 835a 846a 900a 65 903a 916a 930a 65 935a 946a 1000a 65 1003a 1016a 1030a 65 1035a 1046a 1100a 65 1103a 1116a 1130a 65 1135a 1146a 1200p 65 1203p 1216p 1230p 65 1235p 1246p 100p 65 103p 116p 130p 65 135p 146p 200p 65 203p 216p 230p 65 235p 246p 300p 65 303p 316p 330p 65 335p 346p 400p

3. 10/27/2004 Peripheral Display Toolkit3 Solution: Peripheral Displays Provide awareness with min attention Separate from primary task Bus LED Display +Get bus info at a glance +Dynamically manages attention it attracts –Hard to accomplish

4. 10/27/2004 Peripheral Display Toolkit4 Why is creating PDs hard? Need to abstract info to be glance-able Need mechanisms for dynamically managing attention PDs attract: –Deciding attention levels to attract (notification levels) –Displaying info appropriately (transitions) Our contribution: –Peripheral Display Toolkit (PTK) supports these key issues in PD creation

5. 10/27/2004 Peripheral Display Toolkit5 Example PTK Applications Remote Activity –Social Guitar –Audio Monitor –Motion Monitor –Remote Awareness Display Bus Displays –Bus Mobile –Bus LED Instant Messenger Status IM Picture FrameSocial Guitar Bus LED BusMobile Orb showing remote activity

6. 10/27/2004 Peripheral Display Toolkit6 Example PTK Applications Remote Activity –Social Guitar –Audio Monitor –Motion Monitor –Remote Awareness Display Bus Displays –Bus Mobile –Bus LED Instant Messenger Status IM Picture FrameSocial Guitar Bus LED BusMobile Orb showing remote activity

7. 10/27/2004 Peripheral Display Toolkit7 Example PTK Applications Remote Activity –Social Guitar –Audio Monitor –Motion Monitor –Remote Awareness Display Bus Displays –Bus Mobile –Bus LED Instant Messenger Status IM Picture FrameSocial Guitar Bus LED BusMobile Orb showing remote activity

8. 10/27/2004 Peripheral Display Toolkit8 Example PTK Applications Remote Activity –Social Guitar –Audio Monitor –Motion Monitor –Remote Awareness Display Bus Displays –Bus Mobile –Bus LED Instant Messenger Status IM Picture FrameSocial Guitar Bus LED BusMobile Orb showing remote activity

9. 10/27/2004 Peripheral Display Toolkit9 Example PTK Applications +Manage attention with abstraction, notification levels, & transitions +Glance-able +Few lines of code +Simplified code design & code re-use IM Picture FrameSocial Guitar Bus LED BusMobile Orb showing remote activity

10. 10/27/2004 Peripheral Display Toolkit10 Simple Example: Motion Monitor no/low motion medium motion high motion

11. 10/27/2004 Peripheral Display Toolkit11 Outline PTK: Motivation & Examples Managing attention –Abstraction –Notification levels –Transitions Toolkit Architecture & Library Conclusion

12. 10/27/2004 Peripheral Display Toolkit12 Managing User Attention Attention is a crucial issue in design –Significant impact on success –More info can be monitored w/o overburdening Attention management is difficult –Used to designing apps that attract full attention –Attention attracted should match info urgency Literature surveys led to key characteristics of attention management info urgencyuser attention Attention none focal

13. 10/27/2004 Peripheral Display Toolkit13 Attention Management Characteristics info urgencyuser attention From survey of peripheral displays: 1.Abstraction enables lower-attention info consumption 2.Notification levels help determine how much attention to attract based on urgency of info 3.Transitions enable display of info to match urgency

14. 10/27/2004 Peripheral Display Toolkit14 Abstraction Enables lower-attention consumption of info Extracting features or reducing fidelity of info Motion Monitor: replaces video images with general motion amount (high, medium, none) video  abstraction  amount of images motion high motion medium motion no/low motion

15. 10/27/2004 Peripheral Display Toolkit15 Notification Levels Indicate the urgency of info –Low levels = non-urgent info –High levels = more urgent info Motion Monitor: doesn’t show urgent info, so uses low notification levels

16. 10/27/2004 Peripheral Display Toolkit16 More on Notification Levels Attention is (roughly) scalar – PD designers must decide where on the scale displayed info should fall 0%100% attention awareness unconscious

17. 10/27/2004 Peripheral Display Toolkit17 More on Notification Levels Attention Type 1. None 0%100% attention awareness unconscious

18. 10/27/2004 Peripheral Display Toolkit18 More on Notification Levels Attention Type 1. None Notification Level Ignore 0%100% attention awareness unconscious

19. 10/27/2004 Peripheral Display Toolkit19 More on Notification Levels Attention Type 1. None 2. Inattention Notification Level Ignore 0%100% attention awareness unconscious

20. 10/27/2004 Peripheral Display Toolkit20 More on Notification Levels Attention Type 1. None 2. Inattention Notification Level Ignore Change blind 0%100% attention awareness unconscious

21. 10/27/2004 Peripheral Display Toolkit21 More on Notification Levels Attention Type 1. None 2. Inattention 3. Divided Notification Level Ignore Change blind 0%100% attention awareness unconscious

22. 10/27/2004 Peripheral Display Toolkit22 More on Notification Levels Attention Type 1. None 2. Inattention 3. Divided Notification Level Ignore Change blind Make aware 0%100% attention awareness unconscious

23. 10/27/2004 Peripheral Display Toolkit23 More on Notification Levels Attention Type 1. None 2. Inattention 3. Divided 4. Focused Notification Level Ignore Change blind Make aware 0%100% attention awareness unconscious

24. 10/27/2004 Peripheral Display Toolkit24 More on Notification Levels Notification Level Ignore Change blind Make aware Interrupt Attention Type 1. None 2. Inattention 3. Divided 4. Focused 0%100% attention awareness unconscious

25. 10/27/2004 Peripheral Display Toolkit25 More on Notification Levels Notification Level Ignore Change blind Make aware Interrupt Demand action Attention Type 1. None 2. Inattention 3. Divided 4. Focused 5. Focused + interaction 0%100% attention awareness unconscious

26. 10/27/2004 Peripheral Display Toolkit26 Transitions Enable display of info to match urgency Depend on notification level –Ignore: no changes –Change blind: fading, slow motion, tiny updates –Make aware: discrete, abrupt updates –Interrupt: flashing, beeping, vibration –Demand action: interrupt until user does action Motion Monitor: varies pulse speeds for different notification levels

27. 10/27/2004 Peripheral Display Toolkit27 Outline PTK: Motivation & Examples Managing attention –Abstraction –Notification levels –Transitions Toolkit Architecture & Library Conclusion

28. 10/27/2004 Peripheral Display Toolkit28 PTK Architecture 1.Support for managing impact on human attention using abstraction, notification levels, and transitions 2.Simplified code design and code re-use 3.Library of common PD components Output Transition Input Notification Map Abstractor

29. 10/27/2004 Peripheral Display Toolkit29 Motion Monitor Architecture

30. 10/27/2004 Peripheral Display Toolkit30 Motion Monitor Architecture

31. 10/27/2004 Peripheral Display Toolkit31 Motion Monitor Architecture motion = medium

32. 10/27/2004 Peripheral Display Toolkit32 Motion Monitor Architecture motion = medium notification level = change blind

33. 10/27/2004 Peripheral Display Toolkit33 Motion Monitor Architecture motion = medium notification level = change blind

34. 10/27/2004 Peripheral Display Toolkit34 Motion Monitor Architecture 5 lines of logic code

35. 10/27/2004 Peripheral Display Toolkit35 Remote Awareness Display Count # people, How much motion? Phone ring? Voices? Add notification level

36. 10/27/2004 Peripheral Display Toolkit36 Motion Monitor  Remote Awareness Display

37. 10/27/2004 Peripheral Display Toolkit37 Motion Monitor  Remote Awareness Display

38. 10/27/2004 Peripheral Display Toolkit38 Motion Monitor  Remote Awareness Display

39. 10/27/2004 Peripheral Display Toolkit39 Motion Monitor  Remote Awareness Display

40. 10/27/2004 Peripheral Display Toolkit40 Motion Monitor  Remote Awareness Display

41. 10/27/2004 Peripheral Display Toolkit41 Motion Monitor  Remote Awareness Display

42. 10/27/2004 Peripheral Display Toolkit42 Remote Awareness Display 54 – total lines of code

43. 10/27/2004 Peripheral Display Toolkit43 General Architecture Diagram

44. 10/27/2004 Peripheral Display Toolkit44 Library Components Input –audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather, Web page parser, serial port communication

45. 10/27/2004 Peripheral Display Toolkit45 Library Components Input –audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather, Web page parser, serial port communication Output –ticker text, Ambient Orb, Phidgets

46. 10/27/2004 Peripheral Display Toolkit46 Library Components Input –audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather, Web page parser, serial port communication Output –ticker text, Ambient Orb, Phidgets Abstractors –motion, people counting, voices, phone ringing

47. 10/27/2004 Peripheral Display Toolkit47 Library Components Input –audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather, Web page parser, serial port communication Output –ticker text, Ambient Orb, Phidgets Abstractors –motion, people counting, voices, phone ringing Notification –exact match, threshold, contains, degree of change

48. 10/27/2004 Peripheral Display Toolkit48 Transitions PTK provides architectural support for implementing as animations Timing Number of steps Flashing (single, repeated) Tied to output –Depend on output modality –Example transitions: color changing and pulsing (orb); fading, motion, & flashing (ticker text)

49. 10/27/2004 Peripheral Display Toolkit49 Future Work & Conclusion PTK supports the creation of PDs Focuses on managing user attention To do this, uses abstraction, notification levels, transitions Validated with 7 example applications –Supports attention management –Facilitates code re-use Future work –Incorporate interruptibility and local context sensing –Cross-modality animations –PD evaluation support

50. 10/27/2004 Peripheral Display Toolkit50 Questions? For more information & downloads: www.cs.berkeley.edu/projects/io/ptk/ tmatthew@cs.berkeley.edu Thanks to… NSF, Ed De Guzman, Jeff Heer, Scott Klemmer, Berkeley & CMU

51. 10/27/2004 Peripheral Display Toolkit51 What about user context? PDs must 2 sets of attention issues: –User context interruptibility, primary activity, focus of attention –Attention management balancing info urgency and user attention PTK focuses on attention management –User context invasive to sense, not practical to assume PDs have access –Management issues are independent of user context Context input can be considered on a library level (in notification maps)

52. 10/27/2004 Peripheral Display Toolkit52 Related Work AROMA: architectural support for abstraction Context Toolkit: infrastructure for context input and some abstraction Phidgets: library of hardware widgets for input and output Real World Interfaces Toolkit: library of X10 output widgets iStuff: infrastructure for distributed inputs and outputs

53. 10/27/2004 Peripheral Display Toolkit53 Focus Attention Peripheral display + attracting focused attention = contradiction? Nope! –Notifications enable displays to be peripheral when not displaying urgent info –Attention is scalar: PDs should cover entire range 0%100% attention awareness unconscious

54. 10/27/2004 Peripheral Display Toolkit54 PTK Validation Created 7 example applications that illustrate: Supports attention management 3 used abstraction 5 used notification levels and transitions Easier to create 5-55 lines of logic code Facilitates code re-use used same input for 4 displays IM Picture FrameSocial Guitar Bus LED BusMobile Orb showing remote activity

55. 10/27/2004 Peripheral Display Toolkit55 Notification Maps If event value equal to:Set notification level to: no motion ignore med motion change blind high motion make aware (null) interrupt (null) demand action Exact Match Notification Map

56. 10/27/2004 Peripheral Display Toolkit56 Notification Maps Exact match checks event for equality w/specified values Threshold determines if event within specified ranges Contains checks if event contains specified strings Degree of change determines if event has changed by a specified amount since last event

57. 10/27/2004 Peripheral Display Toolkit57 Introduction Peripheral displays (PDs) enable you to monitor many info sources while maintaining a calm environment –But, only calm if designed to manage the attention they attract PDs are hard to create –No existing tools or guidelines for managing human attention Peripheral Display Toolkit (PTK): –Enables easier creation of PDs that manage human attention

58. 10/27/2004 Peripheral Display Toolkit58 Why Peripheral Displays? Provide awareness w/ minimal attention Separate from user’s primary activity Non-focal display not used unless providing peripheral info Important to ubicomp vision of many devices to one user

59. 10/27/2004 Peripheral Display Toolkit59 Peripheral Display Toolkit (PTK) Supports the creation of PDs 1. Focuses on managing user attention 2. Simplifies code 3. Provides library of common components Managers user attention with –Abstraction –Notification levels –Transitions

60. 10/27/2004 Peripheral Display Toolkit60 Characteristics of Attention Management Abstraction of info to be glance-able Dynamically managed attention –Notification levels for deciding how much attention to attract –Transitions for displaying info appropriately

61. 10/27/2004 Peripheral Display Toolkit61 Peripheral Display Toolkit (PTK) Supports the creation of PDs Focuses on managing user attention –Balance info urgency and user attention To do this, uses –Abstraction –Notification levels –Transitions

62. 10/27/2004 Peripheral Display Toolkit62 More on Notification Levels