1. Lecture 14 Pervasive Computing Applications
Wireless Networks and Mobile Systems
Lecture 14 Pervasive Computing Applications

2. Lecture Objectives
Understand characteristics and technical challenges of pervasive computing applications
Understand system middleware and support for context-aware ubiquitous computing
Exemplify pervasive computing applications
Pervasive Computing Applications 2

3. Sources
T. Kindberg and A. Fox, “System software for ubiquitous computing,” IEEE Pervasive Computing, Vol. 1, No. 1, 2002, pp
A. Smailagic and D. Kogan, “Location sensing and privacy in a context-aware computing environment,” IEEE Wireless Communications, Vol. 9, No. 5, Oct. 2002, pp
S.S. Yau, et al., “Reconfigurable context-sensitive middleware for pervasive computing,’’ IEEE Pervasive Computing, Vol. 1, No. 3, 2002, pp
M. Roman, et al., “A middleware infrastructure for active spaces”, IEEE Pervasive Computing, Vol. 1, No. 4, 2003, pp
B. Johanson, et al., “The interactive workspaces project: experiences with ubiquitous computing rooms,” IEEE Pervasive Computing, Vol. 1, No. 2, 2002, pp
Pervasive Computing Applications 3

4. Agenda Characteristics of pervasive computing
System support to enable pervasive computing
Middleware for context-aware pervasive computing applications
Examples of pervasive computing applications
Smart homes/rooms
UI’s Gaia-enabled Active Spaces
Stanford’s Interactive Workspaces project
Experimentation
SLP-based pervasive Pocket TV application
Pervasive Computing Applications 4

5. Pervasive Computing Characteristics (1)
Pervasive computing is:
An environment in which people interact with embedded (and mostly invisible) computers and in which networked devices are aware of their surroundings and peers and are able to provide services or use services from peers effectively
The creation of environments saturated with computing and wireless communication, yet gracefully integrated with human users
Pervasive Computing Applications 5

6. Pervasive Computing Characteristics (2)
A pervasive (ubiquitous) system is characterized by:
Physical integration: integration between computing nodes and the physical world, e.g., a whiteboard that records what’s on
Instantaneous Interoperation: devices interoperate spontaneously in changing environments, e.g., a device changes its partners as it moves or as the context changes
Pervasive Computing Applications 6

7. Non-Examples of Pervasive Applications
Accessing over a phone line from a laptop
Neither physical integration nor spontaneous interoperations
The laptop maintains the same association with a fixed mail server
A collection of wirelessly connected laptops based on IEEE at a conference
Discovery of the local network can be considered as physical integration of laptops with a access point
The physical integration in pervasive computing, however, should involve a part of the environment that has a non-electronic function (e.g., a white board, a cup, etc.)
No spontaneous interoperation is possible without considerable human manual intervention
Pervasive Computing Applications 7

8. Borderline Examples of Pervasive Applications (1)
A smart coffee cup and saucer
Physical integration: The cup is a regular cup (a non-electronic function) that contains sensing, processing and networking elements that let it communicate with the saucer of its state (full or empty, held or putdown, hot or cold)
Instantaneous interoperation: Not satisfied if a specialized protocol is used between the cup and saucer as the owner would not be able to use the coffee cup without the saucer
Pervasive Computing Applications 8

9. Borderline Examples of Pervasive Applications (2)
P2P games
Physical integration: client devices can have sensors that allow physical integration
Instantaneous interoperation: not satisfied if preconfigured components are used in the game
A more convincing example would involve players with generic game playing pieces that let them spontaneously join local games not encountered before
Pervasive Computing Applications 9

10. Borderline Examples of Pervasive Applications (3)
A Web-based object discovery system
Devices (e.g., laptops) have small, embedded web servers that allow objects to be accessed by physical hyperlinks
The user is presented with a web page when it senses an identifier of the object
Physical integration
smart devices can use web functionality to allow advertised objects to be integrated into the physical world as web pages without the need to reconfigure browsers
Instantaneous interoperation
not exactly satisfied since the system requires human intervention to keep it going. The human in the loop changes the browser’s associations to advertised web services
Pervasive Computing Applications 10

11. Examples of Pervasive Applications (1)
A magic mirror that shows personal data and actions of those users who face it in a meeting
Physical integration: the mirror can sense the presence of users in a meeting and can record their actions, in addition to its normal physical function
Instantaneous interoperation: the mirror would interact with the room’s other components the moment you switch it on and would make spontaneous association with all relevant local sources of information about users ….
Pervasive Computing Applications 11

12. Examples of Pervasive Applications (2)
A visitor brings his/her laptop into a meeting room and without manually configuring it in any way uses it to send his presentation to the room’s projector
Physical integration: the projector can be activated from any laptop in the room
Instantaneous interoperation: A laptop can spontaneously interact with the projector and control the presentation with a proper UI
Can be made context-sensitive, e.g., allowing only a particular visitor to do so
Pervasive Computing Applications 12

13. Other examples: Miró display
Cornell experiment: reflections on collective experience
Pervasive Computing Applications 13

14. Other examples: Gust of Presence
Delft Univ. of Technology: Gust of Presence project
Pervasive Computing Applications 14

15. The Semantic Rubicon Little evidence exists to suggest that software
alone can meet ubiquitous
computing challenges,
although it is desirable to
have minimum or no
intervention
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 1, 2002.
The semantic Rubicon is the division between system and human
for high-level decision-making or physical world semantics processing
The division should be exposed in system design and the criteria and
mechanisms for crossing it should be clearly indicated
Pervasive Computing Applications 15

16. System Support Discovery
Requiring a common “syntax and vocabulary” model for specifying services
Service
Type
Mismatched
ServiceType=“photoPrint”
ServiceType=“printing”
Missed
opportunity
ServiceType=“digitalFrame”
Pervasive Computing Applications 16

17. A tuple space or an event service ServiceType=“digitalFrame”
System Support (2)
Interaction
Requiring a “common” interoperation model for components to interact
Event systems – publish, subscribe, and handle events
Tuple space – add, read, and remove tuples in a tuple space
(Standardized)
Data Oriented
Interaction
Service
ServiceType=“Print”
DataType=“JPEG”
ServiceType=“print”
DataType=“JPEG”
A common service:
A tuple space or an event service
ServiceType=“digitalFrame”
DataType = “JPEG”
Pervasive Computing Applications 17

18. System Support (3) Adaptation
Without human intervention for achieving “calm computing” as well as spontaneous interaction
Being able to display or manipulate data or UI from other devices in a heterogeneous environment
Possible solutions include:
Dynamic downloading of mobile code to handle UI and communication (as in Jini) – requiring JVM
Dynamic UI generation/transformation (as in Stanford’s Interactive Workspaces) – requiring a common UI description language to describe UI elements, e.g., “This control is a Boolean toggle”
Another example is to use XML for specifying UI for web services
Pervasive Computing Applications 18

19. Human Interface Adaptation
Use the same high-level UI description language to generate device-specific UIs to control lights and displays in a smart room for (a) a desktop HTML browser, (b) a Java-equipped device, and (c) a Palm device
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 1, 2002.
Pervasive Computing Applications 19

20. System Support (4) Physical Integration Examples: Context aware
Context-awareness support is required to allow an application to access different types of contexts (e.g., locations, identities, QoS conditions) while hiding how the information was sensed or collected
Examples:
“Show me the agenda!” - can be made context-sensitive
“Show me related Web pages for museum exhibits!” – can be based on the location and ids of exhibits sensed through barcode or Infrared beacon
I know
who I am
photographing
I know
you provide
photo printing
services
tag
tag
Pervasive Computing Applications 20

21. Positioning for Location-aware Computing
Outdoor – GPS (Global Position Sensing) normally to within 1-5 m accuracy with differential correction
Indoor
Active Bat (shown below) developed by AT&T Lab Cambridge based on ultrasonic sensors (to within 3 cm accuracy)
CMU-TMI (shown next) developed by CMU based on for location tracking (to within 1-5 m accuracy)
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 1, 2002.
Pervasive Computing Applications 21

22. System Support – Location Tracking in Smart Spaces
CMU has developed and deployed a Triangulation, Mapping and Interpolation (CMU-TMI) algorithm based on for location tracking:
Physical locations of access points are known in advance
A mobile device scans all access points (at least 3) within range to determine their signal and noise strengths
The signal strengths are used to infer the distance between the device and the access points using an approximate relation obtained a priori
Triangulation technique is applied to calculate the physical location
Pervasive Computing Applications 22

23. System Support (5) Programming Framework
A programming framework for pervasive computing can be placed at the application or middleware level:
Application level
Providing only application-level coordination mechanisms, e.g., through a “tuple-space” API
Middleware level
Providing more tightly integrated facilities for achieving context awareness, QoS resource management, and adaptive control.
Determining the user’s task and intention, and facilitating associations between components to assist the user in these activities
Pervasive Computing Applications 23

24. System Support (6) Robustness
Failure is a common case in mobile and pervasive computing
Recovering from failures  always prepared to reacquire lost resources
Expiration-based schemes and soft states
Periodic service advertisement and lease in the “common” data-oriented interaction service
Separating failure-free (logic) from failure-prone (e.g., accessing a service or file while moving) application code
More effective failure handling can be done implicitly by the middleware, e.g., retrying an idempotent operation (binding to a remote file) for a number of times
Security – trust between devices, access control
Pervasive Computing Applications 24

25. Middleware for Pervasive Computing
Context-Aware middleware for pervasive computing must address two “broad” characteristics:
Tradeoff between awareness and transparency
For pervasive applications, environment awareness is key to their effectiveness
Cooperation between development support and run time services
Let developers focus on application logic
Pervasive Computing Applications 25

26. RCSM: A Reconfigurable Context Sensitive Middleware
ASU’s RCSM addresses the following “specific” characteristics:
Uniform development support
providing a uniform way to express context awareness without restricting to a specific language, OS, or environment
Application-specific context acquisition, analysis and detection
allowing developers to express the need for context data in a platform-independent way without knowing “how” data are obtained
Context-triggered actions
transparently invoking actions whenever the specified contexts are valid
Transparent support for ad hoc communication
abstracting the details of ad hoc communication from applications including proactively discovering new devices, establishing new communication links and notifying the application when a device is found
Pervasive Computing Applications 26

27. RCSM: A Reconfigurable Context Sensitive Middleware (2)
ASU’s RCSM provides a context-aware interface description language (CA-IDL) API for an application to
Define contexts used by the application
Specify actions taken based on context values
Compilers are provided to build adaptive object containers (ADCs) based on CA-IDL
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 3, 2002.
Pervasive Computing Applications 27

28. Pervasive Computing Example 1: UI’s Gaia-enabled Active Spaces (1)
Gaia is a distributed middleware infrastructure that coordinates software entities and heterogeneous network devices contained in a physical space
The physical space is extended into an “active space” by adding coordination via a context-based software infrastructure
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 4, 2002.
Pervasive Computing Applications 28

29. Pervasive Computing Example 1: UI’s Gaia-enabled Active Spaces (2)
Event manager service: for event channel registration/notification
Context service: for querying/registering context information such as people’s locations
Presence service: for maintaining soft states of resources including digital entities (applications and services using heartbeats) and physical entities (devices and people using sensors)
Space repository: for displaying/retrieving information about all entities in the active space
Context file system: associating data with context and format info to allow context-sensitive data retrieval presented with the right UI format, e.g., make personal data available to applications conditioned on presence
Example: Presentation Manager Application
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 4, 2002.
Pervasive Computing Applications 29

30. Pervasive Computing Example 1: UI’s Gaia-enabled Active Spaces (3)
EX: Presentation Manager Application
Speaker X with an RF active badge and a handheld walks into a Gaia-enabled room
Presence service detects the badge and sends a “X is here” event with the user profile info of X contained
Space repository receives the event and displays the user info.
Context file system obtains the speaker’s mount point and mounts the speaker’s presentation file ready for access
The handheld (controller) detects the directory server and then sends a heartbeat event to the heartbeat channel
Presence service detects and sends a “new device” found event
Space repository receives the event and displays the handheld info
Both speaker X and handheld are now entities of the active space
A prototype of Gaia-enabled active space
Context: Speaker X enters the room
Action: start the presentation manager application
Read the presentation file
Allowing X to control presentation by control events generated from the touch screen or handheld
Pervasive Computing Applications 30

31. Pervasive Computing Example 2: Stanford’s Interactive Workspaces (1)
Stanford’s Interactive Workspaces is based on a tuple space data model
Event Heap: Allowing name-type-value events (with expiration times) to be posted and retrieved
Data Heap: Allowing data with attributes (e.g., format) posted
iCrafter: Providing service advertisement/invocation and a UI generator that returns the best interface for the device
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 2, 2002.
Pervasive Computing Applications 31

32. Pervasive Computing Example 2: Stanford’s Interactive Workspaces (2)
Room-based cross-platform interfaces
The room-control system stores the geometric arrangement of screens and lights in the room in a configuration file and will automatically provide controllers on any device supporting a UI format available through iCrafter
Java Swing
Palm
html
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 2, 2002.
Pervasive Computing Applications 32

33. Experimentation: SLP-based Pervasive Pocket TV
SLP (RFC 2608) is an IETF standard that provides a scalable framework for automatic resource discovery on IP networks. Three entities are defined in a SLP system:
User agent (UA)
Server agent (SA)
Directory agent (DA)
A UA initiates service discovery on behalf of one or more applications. It can send queries to all SAs via multicast if a DA does not exist or to a DA to discover services via unicast.
A SA works on behalf of one or more services. It responds directly to UA queries via unicast. If a DA exists, a SA can register its services with the DA to expose its services.
Pervasive Computing Applications 33

34. Experimentation: Pervasive Pocket TV (2)
A DA serves as a centralized information repository. It accepts SA registrations and answers UA directory service queries. The DA support is optional and is introduced for performance and scalability considerations.
Application
Service
query UA SA
reply
directory service query
service registration DA
directory service reply
acknowledgment
Pervasive Computing Applications 34

35. Experimentation: Pervasive Pocket TV (3)
DA Discovery: Passive versus Active
Multicast SrvRqst (service:directory-agent)
UA/SA DA
Unicast DAAdvert
Multicast DAAdvert
UA/SA DA
Pervasive Computing Applications 35

36. Experimentation: Pervasive Pocket TV (4)
Three configurations
Small: No DA
Multicast SrvRqst
Multicast SrvRqst SA UA SA
Unicast SrvRply
Unicast SrvRply
Medium: 1 DA
Unicast SrvReg SA
Unicast SrvAck
Unicast SrvRqst UA DA
Unicast SrvRply
Unicast SrvAck SA
Unicast SrvReg
Large: Multiple DA’s SA UA DA SA SA DA UA SA
Pervasive Computing Applications 36