1. Mobile and Location-Based Services Jason I. Hong jasonh@cs.cmu.edu May 04 2007

2. The Big Picture Mobile social computing –inTouch: Coordination for Families and Small Groups –Whisper Mobile: Coordinating groups for social events Large-scale mobile collaboration –Hitchhiking: Estimating “busyness” of places Mobile data –Gurungo: linking desktop and mobile devices Usable privacy and security –Contextual Instant Messaging –People Finder –Grey: Access control to resources Memory support –Memory Karaoke

3. The Big Picture Mobile social computing –inTouch: Coordination for Families and Small Groups –Whisper Mobile: Coordinating groups for social events Large-scale mobile collaboration –Hitchhiking: Estimating “busyness” of places Mobile data –Gurungo: linking desktop and mobile devices Usable privacy and security –Contextual Instant Messaging –People Finder –Grey: Access control to resources Memory support –Memory Karaoke

4. inTouch: Coordination for Families Make it easier to coordinate with others while mobile –Better awareness and messaging Target Users: Small to med. groups of people Fluid and demanding schedule Multiple responsibilities Examples: Dual-career families Work groups Ad hoc (ex. conferences) Carpools Mobility AwarenessMessaging

5. Dual-Career Families Coordination breakdowns inevitable –Children’s activities change without notice –Parent’s meetings run over –Impromptu appointments –Unexpected traffic Result: –High levels of anxiety –Some parents fear about “forgetting” their children Need support for awareness and improvisation

6. inTouch: Coordination for Families Two week field study with six dual-career families

7. Check, Double Check, Triple Check

8. Key Transition Times

9. inTouch: Coordination for Families Make it easier to coordinate with others while mobile –Better awareness –Contextual messaging Combines: Shared calendar Shared todo lists Reminders Real-time location Proximity

10. Project: InTouch It’s 4:30pm and Mom is stuck in traffic inTouch checks her calendar and sees she’s supposed to pick up Cindy from ballet

11. Project: InTouch Mom’s phone senses that she is in a traffic jam, and automatically prepares a status message Mom hits “send”, and Cindy sees that Mom is running late. Cindy decides to wait inside.

12. Contextual Messaging Using current context to: –Select a message template –Fill in the blanks (like a MadLib) –In most cases, can just hit “send” When is contextual messaging useful? –Calendar alarms “running late, will be there in ” –Current activity “in a meeting now, done at ” –Daily rhythms “picked up kid ok” at 3PM –Messages received “where r u?” -> “I am at ”

13. Contextual Messaging Messaging can be linked to calendar or reminders –S: Can you get dinner tonight? –J: Ok, I will pick up __________ on my way home –Activate as a reminder when you leave work Message easy to select around 4PM Fill in the blank based on patterns and what’s near your home

14. Example Mockups Currently developing working prototypes

15. The Big Picture Mobile social computing –inTouch: Coordination for Families and Small Groups –Whisper Mobile: Coordinating groups for social events Large-scale mobile collaboration –Hitchhiking: Estimating “busyness” of places Mobile data –Gurungo: linking desktop and mobile devices Usable privacy and security –Contextual Instant Messaging –People Finder –Grey: Access control to resources Memory support –Memory Karaoke

16. Whisper Mobile Goal: Make it easy to find, share, and coordinate friends going to social events

17. Whisper Mobile: Creating an Event Minimal text input –Use location –Use audio –Use camera

18. Continuing Work Developing working prototype of web site and mobile –Web crawler for finding social events –Web site to coordinate on scale of weeks and days Link with inTouch –Coordinate friends –See who’s late, where we’re going next –Mobile to coordinate on scale of hours and minutes

19. The Big Picture Mobile social computing –inTouch: Coordination for Families and Small Groups –Whisper Mobile: Coordinating groups for social events Large-scale mobile collaboration –Hitchhiking: Estimating “busyness” of places Mobile data –Gurungo: linking desktop and mobile devices Usable privacy and security –Contextual Instant Messaging –People Finder –Grey: Access control to resources Memory support –Memory Karaoke

20. Project: Hitchhiking Most location-based services about where you are Hitchhiking is about the “busyness” of places –“Is the café busy?” –“How long are the lines at the airport?” –“Where’s an empty room?” –Is there any parking at the shopping district?

21. Project: Hitchhiking Estimate number of people in a place by counting the number of wireless devices there Periodically upload count + location to our servers Other people can query our servers

22. Project: Hitchhiking How well does Hitchhiking work?

23. Project: Hitchhiking Privacy? –Upload anonymized counts only –Upload from approved places only –Our server shows “busyness” of a place only Advantages –Cheap, uses existing devices (everyone is a “sensor”) –Deployable, don’t have to set up lots of new sensors –Privacy What’s next? –Map visualizations

25. The Big Picture Mobile social computing –inTouch: Coordination for Families and Small Groups –Whisper Mobile: Coordinating groups for social events Large-scale mobile collaboration –Hitchhiking: Estimating “busyness” of places Mobile data –Gurungo: linking desktop and mobile devices Usable privacy and security –Contextual Instant Messaging –People Finder –Grey: Access control to resources Memory support –Memory Karaoke

26. GurunGo Goal: Make it easy to access useful information while mobile Observation #1: People still tend to print out online maps, despite having mobile device. Why? –Found it via desktop, easier to print than to copy to mobile –Slow or expensive wireless connections –Inconvenient form factor on mobile device Observation #2: People don’t do the same kind of web browsing on mobile phones as on desktops –Don’t have to support all information finding tasks, just ones more likely to be done when mobile

27. GurunGo Scenarios Idea: Tie mobile more closely with desktop You find an interesting product while browsing –Use GurunGo to copy-and-paste to mobile –Augments with product reviews –Copies to mobile –Kept until explicitly deleted As you browse web on desktop: –GurunGo scans HTML for maps –Generates speech-based directions –Copies to mobile –Directions eventually discarded after given time

28. GurunGo Usage Acquire –Let people explicitly copy-and-paste info to mobile –Let people implicitly copy info via regular web browsing GurunGo scans pages seen for potentially useful stuff Augment –Look for known data types, make mobile data more useful –Ex. Augment maps with speech-based directions Copy (to mobile in the background) Browse –Organize data based on common data types –Street addresses, product comparisons, phone #s

29. GurunGo: Speech-based Directions

30. Nice Features of GurunGo Reduces number of clicks to get to useful information –Can support specific information finding tasks while mobile –Currently: Directions, products –Future: Movies, phone #s, dates and times, recent emails Works even if you don’t have wide-area wireless –Works disconnected (no network or don’t want to pay) –Only needs personal area network (Bluetooth)

31. The Big Picture Mobile social computing –inTouch: Coordination for Families and Small Groups –Whisper Mobile: Coordinating groups for social events Large-scale mobile collaboration –Hitchhiking: Estimating “busyness” of places Mobile data –Gurungo: linking desktop and mobile devices Usable privacy and security –Contextual Instant Messaging –People Finder –Grey: Access control to resources Memory support –Memory Karaoke

32. The Problem Mobile devices becoming integrated into everyday life –Mobile communication –Sharing location information with others –Remote access to home –Mobile e-commerce Managing security and privacy policies is hard –Preferences hard to articulate –Policies hard to specify –Limited input and output Leads to new sources of vulnerability and frustration

33. Our Goal Develop core set of technologies for managing privacy and security on mobile devices –Simple UIs for specifying policies –Clear notifications and explanations of what happened –Better visualizations to summarize results –Machine learning for learning preferences –Start with small evaluations, continue with large-scale ones Large multi-disciplinary team and project –Six faculty, 1.5 postdocs, six students –Supported by NSF, CMU CyLab –Roughly 1 year into project

34. The Big Picture Mobile social computing –inTouch: Coordination for Families and Small Groups –Whisper Mobile: Coordinating groups for social events Large-scale mobile collaboration –Hitchhiking: Estimating “busyness” of places Mobile data –Gurungo: linking desktop and mobile devices Usable privacy and security –Contextual Instant Messaging –People Finder –Grey: Access control to resources Memory support –Memory Karaoke

35. Contextual Instant Messaging Facilitate coordination and communication by letting people request contextual information via IM –Interruptibility (via SUBTLE toolkit) –Location (via Place Lab WiFi positioning) –Active window Developed a custom client and robot on top of AIM –Client (Trillian plugin) captures and sends context to robot –People can query imbuddy411 robot for info “howbusyis username” –Robot also contains privacy rules governing disclosure

36. Contextual Instant Messaging Privacy Mechanisms Web-based specification of privacy preferences –Users can create groups and put screennames into groups –Users can specify what each group can see

37. Contextual Instant Messaging Privacy Mechanisms Notifications of requests

38. Contextual Instant Messaging Privacy Mechanisms Social translucency

39. Contextual Instant Messaging Privacy Mechanisms Audit logs

40. Contextual Instant Messaging Evaluation Recruited ten people for two weeks –Selected people highly active in IM (ie undergrads ) –Each participant had ~90 buddies and 1300 incoming and outgoing messages per week Notified other parties of imbuddy411 service –Update AIM profile to advertise –Would notify other parties at start of conversation

41. Contextual Instant Messaging Results Total of 242 requests for contextual information –53 distinct screen names, 13 repeat users

42. Contextual Instant Messaging Results 43 privacy groups, ~4 per participant –Groups organized as class, major, clubs, gender, work, location, ethnicity, family –6 groups revealed no information –7 groups disclosed all information Only two instances of changes to rules –In both cases, friend asked participant to increase level of disclosure

43. Contextual Instant Messaging Results Likert scale survey at end –1 is strongly disagree, 5 is strongly agree –All participants agreed contextual information sensitive Interruptibility 3.6, location 4.1, window 4.9 –Participants were comfortable using our controls (4.1) –Easy to understand (4.4) and modify (4.2) –Good sense of who had seen what (3.9) Participants also suggested improvements –Notification of offline requests –Better notifications to reduce interruptions (abnormal use) –Better summaries (“User x asked for location 5 times today”)

44. Contextual Instant Messaging Current Status Preparing for another round of deployment –Larger group of people –A few more kinds of contextual information Developing privacy controls that scale better –More people, more kinds of information

45. The Big Picture Mobile social computing –inTouch: Coordination for Families and Small Groups –Whisper Mobile: Coordinating groups for social events Large-scale mobile collaboration –Hitchhiking: Estimating “busyness” of places Mobile data –Gurungo: linking desktop and mobile devices Usable privacy and security –Contextual Instant Messaging –People Finder –Grey: Access control to resources Memory support –Memory Karaoke

46. People Finder Location useful for micro-coordination –Meeting up –Okayness checking Developed phone-based client –GSM localization (Intel) Conducted studies to see how people specify rules (& how well) See how well machine learning can learn preferences

47. People Finder Machine Learning Using case-based reasoning (CBR) –“My colleagues can only see my location on weekdays and only between 8am and 6pm” –It’s now 6:15pm, so the CBR might allow, or interactively ask Chose CBR over other machine learning –Better dialogs with users (ie more understandable) –Can be done as you go (rather than accumulating large corpus and doing post-hoc)

48. People Finder Current Work Small-scale deployment of phone-based People Finder with a group of friends –Still needs more value, people finder by itself not sufficient –Trying to understand pain points on next iteration Need more accurate location –GSM localization accuracy haphazard Integration with imbuddy411 –Smart phones expensive, IM vastly increases user base

49. The Big Picture Mobile social computing –inTouch: Coordination for Families and Small Groups –Whisper Mobile: Coordinating groups for social events Large-scale mobile collaboration –Hitchhiking: Estimating “busyness” of places Mobile data –Gurungo: linking desktop and mobile devices Usable privacy and security –Contextual Instant Messaging –People Finder –Grey: Access control to resources Memory support –Memory Karaoke

50. Grey – Access Control to Resources Distributed smartphone-based access control system –physical resources like office doors, computers, and coke machines –electronic ones like computer accounts and electronic files –currently only physical doors Proofs assembled from credentials –No central access control list –End-users can create flexible policies

51. Grey Creating Policies Proactive policies –Manually create a policy beforehand –“Alice can always enter my office” Reactive policies –Create a policy based on a request –“Can I get into your office?” –Grey sees who is responsible for resource, and forwards Might select from multiple people (owner, secretary, etc) –Can add the user, add time limits too

52. Grey Deployment at CMU 25 participants (9 part of the Grey team) Floor plan with Grey-enabled Bluetooth doors

53. Grey Evaluation Monitored Grey usage over several months Interviews with each participant every 4-8 weeks Time on task in using a shared kitchen door

54. Grey Surprises Grey policies did not mirror physical keys –Grey more flexible and easier to change Lots of non-research obstacles –user perception that the system was slow –system failures causing users to get locked out –need network effects to study some interesting issues Security is about unauthorized users out, our users more concerned with how easy for them to get in –never mentioned security concerns when interviewed

55. Grey Current work in Visualizations

56. The Big Picture Mobile social computing –inTouch: Coordination for Families and Small Groups –Whisper Mobile: Coordinating groups for social events Large-scale mobile collaboration –Hitchhiking: Estimating “busyness” of places Mobile data –Gurungo: linking desktop and mobile devices Usable privacy and security –Contextual Instant Messaging –People Finder –Grey: Access control to resources Memory support –Memory Karaoke

57. Memory Karaoke Phone-based system for preventing cognitive decline –Take pictures with camera phone –Tag with location, time –Tell stories about them

58. Summary Mobile social computing Large-scale mobile collaboration Mobile data Usable privacy and security Memory support Jason I. Hong jasonh@cs.cmu.edu NSF DARPA Microsoft SenseMap Motorola Nokia

60. Lots of Large-Scale Mobile Apps Gawker Stalker

61. Lots of Large-Scale Mobile Apps One-way Matchmaking

62. Grey Results of Time on Task of a Shared Kitchen Door