Pervasive Location-aware Computing Applications and Issues Professor Lionel M. Ni Dept. of Computer Science Hong Kong Univ. of Science & Technology

2 Outline Motivation and Background Pervasive computing Location-aware computing Applications Technical challenges Conclusions

3 Evolution of Computing Devices Supercomputer Mainframe Server Workstation PC PDA Watch Smart Card RF Chips ??? $10,000,000 $1,000,000 $100,000 $10,000 $1,000 $100 $10 $1 $0.1 $0.01

4 Technologies Drive New Thing Natural technology trends Computation is becoming essentially free Communication is becoming ubiquitous Smart devices Huge numbers of computing devices in the world What are we doing with them? Modes of operation Programs controlling other programs in our environment Human-in-the-loop: computing should be only as visible as I desire; no more, no less...

5 Next Wave Computing Artificial intelligence? Anywhere and anytime services All are interconnected Benefit from computers without any knowledge of computers Making computers invisible Making networks invisible Computers can act intelligently upon and within the environments

6 Trend: Pervasive Computing Mutual dependency and interaction between computers and environments Computers Obtain the information from the environment Utilize the information to dynamically build models of computing Environments Detect other computing devices entering it Challenge: how to teach a computer about its environment?

7 Computing Input Output Processing

8 Network Networked Computing Processing + communication Processing + communication Processing + communication Processing + communication

9 Pervasive Computing Network Processing + communication Processing + communication Processing + communication Processing + communication Location information Sensors Actuators Resource information Environmental Context

10 Pervasive Computing Shorthand for the strongly emerging trend toward: Numerous, casually accessible, often invisible computing devices Frequently mobile or imbedded in the environment Connected to an increasingly ubiquitous network structure The aim is for easier computing, more available everywhere it's needed

11 Ubiquitous Computing Integration of large-scale mobility with the pervasive computing functionality Any computing device, while moving with us, can build incrementally dynamic models of its various environments and configure its services accordingly Proactively build up services in new environments Remember past environments when we re- enter

12 Fine-Grained Networking The next explosion in growth of the Internet will come from connecting to billions of cheap, low-power sensors, effectors, and “smart things” Cybernetic servants in the Internet will interpret and respond to the environment, such as location, gestures, and movement Connectivity will be wireless Transceivers must be comparably small, cheap, and low-power

13 Environmental Context Makes use of different kinds of information User Identity Location Social “In a meeting” Physical Temperature Light and noise levels Computational Network bandwidth CPU speed Display size Temporal Geographic

14 Environmental Context Environmental features are not created explicitly for input to the system Help for more intelligent behavior Hard to give detailed instructions in mobile environment Interfaces must be more intuitive Mobile environment is much more variable than desktop

15 Examples Take your current environment into account in making decisions Turn off cell phone when you enter the lecture hall. When you ask where to go for a meal, notes that it is morning and you are in Hong Kong before making a recommendation. Know who wrote on the whiteboard so a copy of the ink can be ed to the author. Play music you like when you enter an empty elevator. Notify your doctor when your heart rate goes too high.

16 Example: More on Cell Phone How should a cell phone respond to an incoming call Noisy environment in street  ring loudly Quiet environment in office  ring gently Inside a cinema  no ring, but vibrate Attending classes  switch to voic Manual switching is possible but undesirable

17 More Examples Proactive is important in pervasive computing Need some event observation/notification mechanism Ask for autograph... Meeting EventEnter Area Event Shoes You wanted to buy shoes...

18 Other Context-aware Behavior Present information and services proactively Your wearable sunglass shows you the names and the research interests of the people around Execute service automatically When a sensor you wear senses that you feel cold, it turns on the heater automatically Tag “ context ” for later retrieval When attending a lecture, your computer detects that you feel sleepy and starts recording. Later It can show you parts of the lecture that you have missed

19 Why Indoor Location Sensing?

20 Location-aware Computing Motivation location-based action nearby local printer, doctor nearby remote phone directions/maps location-based information real person’s location history/sales/events virtual walkthrough story of city augmented touring machine

21 Location-aware Computing Location-aware services allow to offer value-added service to the user, depending on their current geographic position and will be a key feature of many future mobile applications Many scenarios in pervasive computing Navigation Resource discovery Embedded applications, sensor systems Monitoring and control applications

22 Location Sensing Techniques GPS: used in outdoor Indoor location sensing criteria Accuracy Granularity Latency Direction of movement Power consumption Wiring constraints Physical packaging Regulation constraints Cost

23 GPS The most popular option of obtaining location information today. 24 satellites controlled by US Need to be exposed to 4 satellites GPS lets suitably equipped receivers determine their position within 3-5 meters. It is generally not workable indoors.

24 GPS Trimble GeoXMThe Casio Pathfinder GPS watch

25 Mobile Commerce

26 Mobile Commerce Applications Retails Shopping malls Health care system Manufacturing Inventory tracking Mobile IP phone Your imagination ….

27 Retail Stores & Shopping Malls Customer: enjoy the convenience and fun Easily find the direction to the target items/stores Find and locate your friends if they nearby (can even setup to get notified if they are close-by) Retailers: better management and customer relationship Easily get information to the customer, like special deals/coupons for items/stores nearby More efficient management, e.g. just-in-time customer service Establish better relationship with customer through the provision of better services and experience

28 M-commerce Illustration Direction to target places, or pinpoint location of friends Help manage customer service employees Click for details info or text chat Initiate a voice chat with friends or for customer service person

29 Desk Refrigerator Desk Sofa Refrigerator 2. Tracking movement 3. Notify where you are (Location sensing) 4. Notify your eating schedule 7. Time sensing 8. immobility sensing 11. Proximity sensing 16. Notify “ ok to eat ” 1.walk 5.Stop eating 6.Back to desk 10. Walk around 9. Notify to move 12. Walk away from sofa 13. Distance& Time sensing 14. Notify to stop 15. Go back to desk 17. Go to kitchen 18. Refer Healthy food 19. Eat matched food

30 Other Applications Health care system Tracking patients with wheelchairs Dietary control Manufacturing Auto industry Inventory tracking

31 SWIM Project Infrastructure Sensing Network Locate a mobile user or object within certain accuracy and granularity WIreless Network Allow the communication between mobile devices and the resource-rich Internet/Intranet Mobile Ad-hoc Network and routing: How mobile devices communicate through close-by in-range mobile device when it’s out of the range from the Wireless Access Point (WAP)

32 SWIM Prototype

33 Mobile Ad-hoc Network Ad-hoc communication & routing Allow mobile devices to communicate with each other in the absence of WAP Enable mobile devices to communicate with the wired-network/far-away devices through peers forwarding VoIP between handheld devices Allow hands-free communication on handheld devices

34 Who Are the Players? The design of good location-aware computing systems cuts across many areas of CS/EE E.g., sensors, signal processing, networking, mobility, data management, graphics/visualization, planning, HCI, … Interdisciplinary Research Explore new business models Establish new customer behavior analysis Develop new applications for m-commerce

35 Fundamental Challenges Collecting sensor data Scalability, heterogeneity Raw format Measurement error, noisy environment Processing sensor data into “context” Adjust format, correct for error Aggregate for higher-level context Disseminating to interested applications Scalability, heterogeneity Securing access to secret and private data

36 More Challenges In-door location sensing techniques Resolution and range proportionate to size and mobility of objects Most applications want relative location, not latitude and longitude Commodity components: Berkeley MOTE? Multi-hop ad hoc networking Infrastructure mode and ad-hoc mode IP address assignment, Name and IP mapping Service discovery Low-power multi-hop ad hoc routing

37 More Challenges: Context We are constantly receiving information Problems Information only received once or twice Not received when and where we need it Large amount of distributed data Data movement, storage, retrieval, etc. Possible solutions Place information into the context in which it will be most useful Devices accept and/or deliver information Distributed query processing

38 Software Infrastructure Need a software infrastructure to put all those together Software models Standard middleware support Service discovery Security and privacy issues More intuitive HCI

39 Current Pervasive Machines Still Mostly “Modern” More, faster, refined: RAM Processor power Network bandwidth Disk storage Windows Icons Menus Pointers Wireless networks Portable Pen Input

40 User Interface Trends Beyond Twentieth Century “Modern” Passive Nominal Individual Single machine Local data Fixed External Perceptual/Reactive Personal Collaborative Distributed Global data Portable/wearable Implanted (medical)

41 Into the Twenty-first Century... Wireless networking Extreme portability Longer battery life Perceptual interfaces Sense: sounds, video images Recognize: words, people, gestures, gaze Respond: process commands, access information

42 e911 Scenario Assumptions John, a physician, volunteers to help patients and carries a PDA Bill, over 70, has heart disease and carries a PDA with his disease history and other information PDA: cell phone, wireless LAN/WAN, Bluetooth Both John and Bill are in a shopping mall and Bill has heart attack. Bill pushes one button on his PDA PDA calls 911 and e911

43 Challenges How is people’s mobility supported? How are Bill’s location and John’s location known and integrated with the medical service (mobile service)? How can the system provide John’s expertise while keeping his privacy? How is Bill’s medical information guaranteed to be secure and not revealed to the shopping mall? How are practical jokes protected? How are service requests made non-repudiate? How are quacks excluded? Etc. One solution: SPLENDER (PerCom 2003)

44 Conclusions Most of the exciting stuff will happen in the next few years! Many technical challenges Commercial and economical impacts New business models Security and privacy issues Pricing and charging models Social impacts

45 Questions?