Fasten the seat belt, here we Go

Home sweet home, where the lights along the corridor flicker

3500°F !! The 3500 ° F Incinerator for the Charcoal Grill

The Duel Between Stew and IceCube Hey, Who Turns Off The Light?

Attention! Time for Roll Call!

Does Your Smart House Know Better?

A Safety-Oriented Programming Model for Pervasive Computing Systems Pervasive & Mobile Computing Lab Hen-I Yang April 21, 2007

Characteristics of Pervasive Computing Systems Interaction with Physical World Highly Personalized/Customized Dynamicity Open Diversity/Heterogeneity Complexity and Scalability Issue Predication on Human Intention Multiple Modalities in Interface

Proposed New Model Safety Vs. Expressiveness

Interrupt & Exception Handling Memory access violation Y = X/0 Divided by 0 Interrupt Fire Alarm Impermissible Context Out of Op Range Exception

The Four Elements

Elements in Pervasive Computing Environment Sensor Actuator Service Applications User Space System Support Devices Services Users Space

Element 1: Device Sensors, actuators and other smart appliances are the I/O devices that serve as the bridge between the physical world and the pervasive computing systems. Source of data and/or sink of commands Accurate description of devices is the key, it defines how to interact with these devices

Modeling of Device Service oriented architecture Can be interpreted on the fly, or created and compiled prior Type: (Sensor/Actuator/Both) Domain of data: e.g. Temperature, Luminance, Device Internal Data Schema: XML Document Type Declaration Including, but not limited to maximum and minimum values, resolution of data, unit of data Interface Methods: Methods available to be invoked externally Physical Medium Description: Description of the Interface/ Conduit/Channel that is to be used Interaction Protocol Description: Support for Poll or Push methods, maximum frequency of data exchange Other Support Information: e.g. Vendor and other contact/support information

Safety Measure for Device Automatically appended exception handling routine Device Safety: Avoid conflicting directives and unsafe/unacceptable operations, similar to Electric Breaker Regulate the incoming commands and detect abnormal command/access pattern Using maximum and minimum values to filter or red-flag abnormal values

Element 2: Service The core active member of pervasive computing system More complicated than the usual software in desktop because of the heterogeneity and dynamicity Service/Application Service Lifecycle management is crucial Implemented Aligned (Target Setup, possibly using user preferences) Binded (to devices or other services) Context input as conditions Logics/Actions/Behaviors Change of Preference Change of Binding Change of Conditions Priority Level

Active Active Contexts Modeling of Services Terminated Aligned Binded User Preference Service Behavior 1 Service Behavior 2 Service Behavior n …. Service Binded Service Registry Services Programmed Services Devices Active Context Change Event Service/Device Change Event User Preference Change Event Impermissible Context Event

Safety Measure for Service Service Safety: Whenever device safety, user safety or space safety is violated, the exception handler will suspend or terminate all relevant services Services have to properly Binded and Aligned before become eligible for execution. The existence of Terminated state ensure each service has a Hand Break that can be activated when needed Throughout execution, services subscribe to relevant active contexts, and suspend or terminate when impermissible contexts become active Service is as safe as the safety feature of the language used to implement it, but the safety measure reduce the risk of damage caused by services

Element 3: User In every single pervasive computing system, even those aiming at automation without human interventions, the ultimate goal: to achieve and fulfill functionality defined by users’ desires and intentions. There will be no need for the existence of any system if there was nothing users want to achieve. Integral part of pervasive computing system, as their behaviors or mere existence inadvertently affect the system

Modeling of User Digital representation: User Profile Static profile Describes users’ relevant attributes and preferences Used to setup the goal of the service and applications Critical for customization and users’ intention prediction E.g. special assistance needed, or the preferred room and water temperature settings, or even users’ calendar Dynamic profile Data gathered by the system about the user Updated by the system Treated as contexts in the system E.g. the current location of the user, the blood pressure and glucose level attained by medical sensors

Safety Measure for User 1 st and 2 nd Laws of Robotics A robot may not injure a human being or, through inaction, allow a human being to come to harm. A robot must obey orders given it by human beings except where such orders would conflict with the First Law User Safety 1: Smart space should monitor users’ status and activate emergency measure should users appear to be in danger [using the mechanism for Space Safety] User Safety 2: Smart space should function to serve users’ preference and obey users’ order [using the mechanism for service safety] Humans need to be locked up so they cannot commit crime to harm other humans

Element 4: Space Most models do not consider space itself as a critical element of pervasive computing system Influence the decision making (context) Big pictures on the effects of service and applications Big pictures for user apprehension and visualization Provide critical perspective when there are resource sharing

Modeling of Space Independent Context Manager and Context Graph Contexts defined using standardized ontology Categorization of preferred, transitional and impermissible contexts Visualization in a summary-in-a-glance view on overall context of the space Hot Indoor Warm Indoor Cold Indoor Overcooked Steak Cooked Steak Raw Steak Murky air Clean air Context Potential Transitions Preferred Context Impermissible Context Blown fuse Medium power draw Low power draw

Safety Measure for Space Space Safety Monitor: Whenever impermissible context is detected, the space safety monitor will invoke exception handling routine, similar to interrupt/signal capture In the worst case where there is no exception handling available, a prompt would be send to the user or the emergency contact person, so human actions can take place to handle the situation

Recap Enforce Safety at Each h/w and s/w Entity in the intelligent environment Device Safety Enforcer Service Signal Handling Vector Respect Users’ Wishes, and Constantly Look After Them Look at the Big Picture, and make sure the interactions among users and entities are not harmful

Prior Endeavors

ATLAS Middleware

IDE: Atlas Plug-in for Eclipse

What does Atlas plug-in do? Browse Available Entities and Services Provide Information on OSGi bundles Design by Selection Semi-automatic Environment Setup Integrate seamlessly with all the nice features and assistance provided by Eclipse Deploy New Bundles Back to Smart Space

Context-Driven Model Reactive Programming Model Ontology Based context graph Categorize contexts into desirable, transitional and impermissible context Find the actions required to take us from currently active contexts to preferred contexts, and activate special measures when impermissible contexts are active

Hybrid Model

Epilogue

Past, Present and Future Past Establish safety-oriented programming model Implement IDE to support basic OSGi bundle authoring Present Implementation of the new safety framework with safety enforcer into both the ATLAS Middleware and the IDE support Future Evaluating the safety feature of the intelligent environment created based on this framework

Conclusion Pervasive Computing Systems are Intimate The failure and unreliability can result in strong feeling of betrayal and frustration Intelligent Environment Affects Numerous Aspects of the Daily Life Ensure Safety of the System by Ensuring Safety of All Entities and the Environment as a Whole Safety Enforcement is accomplished by the collaboration at both implementation/ compilation time as well as runtime.

Thank You

Design Philosophy Safety Feature: Each individual device has a device breaker to monitor abnormal access pattern and invalid values The smart space as a whole guarded by context manager Orthogonal Elements: Simplify the design, implementation and maintenance of the heterogeneous and dynamic system Dual text/object approach: Text based implementation allows common, configurable services to be created without coding and maximum portability; Object embodies more complicated or less-common logic to be created for specific services

The Path Forward Analysis of Pervasive Computing Systems and elements of them Design programming model and necessary abstractions to simplify design and implementation Focuses on safety features to protect users, devices and the smart space as a whole

Modeling of Services Programmed Service Aligned Service Binded Service User Preference Service Registry Services Devices Active Contexts Service Behavior 1 Service Behavior 2 Service Behavior n …. Active Context Change Event Service/Device Change Event User Preference Change Event Service Binded

Pervasive Computing “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” -- Mark Weiser “The creation of environments saturated with computing and communication yet gracefully integrated with human users” -- Pervasive Magazine, Inauguration Issue “Embedding computation into the environment and everyday objects to enable people to interact with information- processing devices more naturally and casually than they currently do, and in whatever location or circumstance they find themselves.” Other alternative terms: Ubiquitous computing, calm technology, things that think, everyware -- Wikipedia

Goals of Proposed Programming Model Safe Service Oriented Architecture (SOA) Open Configurable/Customizable Orthogonal Description Easily Programmable

Background

ATLAS Architecture Applications Intelligent Environment (e.g. Smart Home) ATLAS Sensor/Actuator Technology Programming Tool

ATLAS Sensor Platform The basic building block for programmable pervasive spaces. Atlas provides physical nodes for connecting various heterogeneous devices, a system for translating those devices into software services, a system for maintaining a library of device services and their interfaces, and a runtime environment for accessing services and composing applications.

Middleware Atlas Developer APIContext Manager Originally designed and implemented for Context- Driven Model Implemented using ontology software and inference engine Parts of implementation concerning impermissible contexts and their emergency handling routines are retained and modified Visualization will be added to provide quick snapshot view MethodClass/InterfaceDescription receivedDataAtlasClientData handler called by service bundle when data arrives addPropertyAtlasServiceAdd a property pair (key, value) to be associated with this service removeProp erty AtlasServiceRemove a property associated with this service getPropertie s AtlasServiceGet all properties associated with this service sendComma nd AtlasServiceSend control commands (used by service associated with an actuator) getDataAtlasServicePull data from sensor associated with this service SubscribeAtlasServiceRequest data stream from sensor UnsubscribeAtlasServiceHalt data stream from sensor isSubscriberAtlasServiceCheck if application is receiving data stream from particular sensor dataHandlerAtlasService Data handler called by middleware when data arrives for particular service. Must be implemented by device service developer.