Presentation is loading. Please wait.

Presentation is loading. Please wait.

BlindSight: Eyes-free mobile phone interaction Kevin Li, University of California, San Diego Patrick Baudisch, Microsoft Research Ken Hinckley, Microsoft.

Similar presentations


Presentation on theme: "BlindSight: Eyes-free mobile phone interaction Kevin Li, University of California, San Diego Patrick Baudisch, Microsoft Research Ken Hinckley, Microsoft."— Presentation transcript:

1 blindSight: Eyes-free mobile phone interaction Kevin Li, University of California, San Diego Patrick Baudisch, Microsoft Research Ken Hinckley, Microsoft Research

2 calendar preview “Monday 9am” “tic, tic, sssssh” “How about Monday morning?” “Yeah, looks like I’m free after 10” blindSight

3 is an application running on Microsoft Windows Smartphone is launched when user places or receive a call. It then replaces the in-call menu unlike the in-call menu, blindSight uses auditory feedback

4 why?

5 PCs… PC screens have the users’ undivided attention  design for the visual channel

6 eyePhone

7 environment

8 visual impairment

9 screen-less device

10 can’t see screen

11

12

13 Lots of information is stored on mobile phones…

14 … the interfaces are visual

15 # of participants “I need to access as part of a phone conversation:” survey

16  Ok, so let’s just translate all text from visual to auditory –“Menu: Press 1 to search contacts; press 2 to add a contact; press 3 to access your calendar…”  Wait, that sounds familiar

17 Please listen carefully as our options have changed…

18 related work

19  User’s should be able to “dial ahead” [Perugini et al.,CHI 2007]  Zap and Zoom allows users to jump to locations using shortcuts [Hornstein, UBILAB Rep 1994]  Use visual channel to inform users about options [Yin and Zhai, CHI 2006] interactive voice response

20  Time compress audio [Dietz and Yerazunis, UIST 2001]  Integrate speech commands into the conversation [Lyons et al., CHI 2004] phone interaction mid-conversation

21 blindSight’s auditory feedback

22 audio is heard only by the user, not by the person at the other end

23 rationale  people can recover from audio interruptions as long as interruption is short human-human conversation contains redundancy can we use this redundancy to inject auditory feedback from the device?

24 how do we make sure device feedback fits into these time windows of low information content?

25 rules 1. feedback only on-demand hear voice note mute speaker phone hear task list add contact record voice find contact cal endar hear s hear text message home

26

27

28 rules 2. brevity delete abc def tuv ghi wxyz pqrs mno jkl next play find contact type 6 “200 hits” type 2 “12 hits” type 7 “Marion”

29 rules 3. non-speech previews of composites week day 3 hours ½ hour block ½h preview day + – – – + _ whereAmI go today preview 3 hours ++ calendar (what if the content is a long list, such as appointments for a day?)

30

31

32 rules 4. decomposition week day 3 hours ½ hour next preview day + – – – + _ whereAmI go today preview 3 hours (what if the content is a long list, such as appointments for a day?)

33 rules 5. interruptability user interface runs as a separate thread

34 action deletesave rules 6. minimize modes action deletesave mon wed tue sat thu fri sun action deletesave pick daystart timeend time … modes

35 rules 6. minimize modes (  avoid wizards) week day 3 hours ½ hour block ½h preview day + – – – + _ whereAmI go today preview 3 hours

36 hear voice note mute speaker phone hear task list add contact record voice find contact cal endar hear s hear text message home

37 add contact save delete

38 iterator action deletesave menu patterns

39 calendar week day 3 hours ½ hour block ½h preview day + – – – + _ whereAmI go today preview 3 hours

40 (shows fast usage by an experienced user) demo video

41

42

43 hardware

44

45

46

47 space

48 epoxy dots enlarged spaces

49 Flip Ear Visual error

50

51 # * 0.

52 Flip Ear Visual error

53 blindSight evaluation

54 interfaces Smartphone 2003 (sighted)BlindSight (eyes-free) vs.

55 task while “driving”idle (1) schedule appointments and (2) add contacts

56 Was not missing information Knew position in the menu Knew what day/time I was at Felt in control of the conversation Better for setting meeting times Prefer if driving and talking Prefer Overall blindSightSmartphone Overall preference results

57 1. brevity is good, but use in moderation clarification of navigation overrides brevity 2. predictable/modeless user interface is key 3. auditory feedback goes a long way even during phone call (disclaimer: need to study how it interferes with activities… driving) lessons

58 environment visual impairment can’t see screen screen-less device next:

59 eyePhone

60 eyesFreePhone ?

61 blindSight: Eyes-free mobile phone interaction Kevin Li, University of California, San Diego Patrick Baudisch, Microsoft Research Ken Hinckley, Microsoft Research

62

63 extra slides

64

65 1. built a system 2. a set of eyes-free design rules 3. keypad modifications enabling eyes-free 4. user study comparing with a product (Smartphone 2003) contributions

66 1. feedback only on-demand 2. brevity 3. non-speech previews of composites 4. decomposition 5. interruptability 6. minimize modes rules

67 iterator action deletesave menu patterns

68 hear voice note mute speaker phone hear task list add contact record voice find contact cal endar hear s hear text message home

69 add contact save delete

70 iterator action deletesave menu patterns

71 calendar week day 3 hours ½ hour block ½h preview day + – – – + _ whereAmI go today preview 3 hours

72 hear voice note mute speaker phone hear task list add contact record voice find contact cal endar hear s hear text message save delete week day 3 hours ½ hour block ½h preview day + – – – + _ whereAmI go today preview 3 hours delete abc def tuv ghi wxyz pqrs mno jkl next play add contact find contact calendar home type folder n items item play + – – – + _ + preview + , tasks, voice, SMS home help hold bottom left for hold bottom right for menu

73 …is a phenomenon in which people who are perceptually blind in a certain area of their visual field demonstrate some visual awareness, without any qualitative experience blindSight... [wikipedia]

74 don’t mode me in blind sight 10 design rules to allow eyes-free use and flow tactile features

75 …are in in a mobile situation If they requires visual attention, users will fail at their current activity interference with social activities drive off the road… phones…

76

77 vs. baseline interfaces

78 task times


Download ppt "BlindSight: Eyes-free mobile phone interaction Kevin Li, University of California, San Diego Patrick Baudisch, Microsoft Research Ken Hinckley, Microsoft."

Similar presentations


Ads by Google