1. SAJAL K. DAS CReWMaN LOCATION-AWARE RESOURCE MANAGEMENT IN SMART HOME ENVIRONMENTS Sajal K. Das, Director Center for Research in Wireless Mobility & Networking (CReWMaN) Department of Computer Science and Engineering (CSE@UTA) The University of Texas at Arlington, USA E-mail: das@cse.uta.edu URL: http://crewman.uta.edu [Funded by US National Science Foundation]

2. SAJAL K. DAS CReWMaN What is a Smart Environment ? Saturated with computing and communication capabilities to make intelligent decisions in an automated, context-aware manner  pervasive or ubiquitous computing vision. Technology transparently weaved into the fabric of our daily lives  technology that disappears. (Weiser 1991) Portable devices around users networked with body LANs, PANs (personal area networks) and wireless sensors for reliable commun. Environment that takes care of itself or users  intelligent assistants provide proactive interaction with information Web. Examples: Smart home, office, mall, hotel, hospital, park, airport

3. SAJAL K. DAS CReWMaN

4. SAJAL K. DAS CReWMaN Smart/Pervasive Healthcare  Consider a heart attack or an accident victim  Desired actions  Coordinate with the ambulance, hospital, personal physician, relatives and friends, insurance, etc.  Control the traffic for smooth ambulance pass through  Prepare the ER (Emergency Room) and the ER personnel  Provide vital medical records to physician  Allow the physician to be involved remotely …  On a Timely, Automated, Transparent basis  PICO (Pervasive Information Community Organization) http://www.cse.uta.edu/pico@cse M. Kumar, S. K. Das, et al., “ PICO: A Middleware Platform for Pervasive Computing, ” IEEE Pervasive Computing, Vol. 2, No. 3, July-Sept 2003.

5. Heart attack victim Pervasive Healthcare Ambulance Ambulance Victim- Ambulance Community Larger community to save patient Physician Hospital Cardiac Surgeon Nurse Spouse Police Traffic control Insurance Co.

6. SAJAL K. DAS CReWMaN PICO Framework  Creates mission-oriented, dynamic computing communities of software agents that perform tasks on behalf of the users and devices autonomously over existing heterogeneous network infrastructures, including the Internet.  Provides transparent, automated services: what you want, when you want, where you want, and how you want.  Proposes community computing concept to provide continual, dynamic, automated and transparent services to users.

7. SAJAL K. DAS CReWMaN PICO Building Blocks  Camileuns (Physical devices) (Context-aware, mobile, intelligent, learned, ubiquitous nodes)  Computer-enabled devices: small wearable to supercomputers  Sensors, actuators, network elements  Communication protocols Camileuns Access point Internet Gateway Access point Gateway Bluetooth 802.11b Cellular …

8. SAJAL K. DAS CReWMaN PICO Building Blocks  Delegents (Intelligent Delegates)  Intelligent SW agents and middleware  Location/context-aware, goal-driven services  Dynamic community of collaborating delegents  Proxy-capable: exist on the networking infrastructure  Resource discovery and migration strategies  QoS (quality of service) management Community Delegents

9. SAJAL K. DAS CReWMaN Visitor’s Delegent Camileuns + Delegents = Chameleons Surveillance Traffic Monitor Information Kiosk Police Community Automobile Community Streetlamp

10. SAJAL K. DAS CReWMaN PICO Architecture PICO Middleware Services Community Delegents Camileuns Access point/ Gateway Access point/ Gateway Bluetooth 802.11b Cellular …

11. SAJAL K. DAS CReWMaN Smart Homes: Objectives  Use smart and pro-active technology  Cognizant of inhabitant’s daily life and contexts  Absence of inhabitant’s explicit awareness  Learning and prediction as key components  Pervasive communications and computing capability  Optimize overall cost of managing homes  Minimize energy (utility) consumption  Optimize operation of automated devices  Maximize security  Provide inhabitants with sufficient comfort / productivity  Reduction of inhabitant’s explicit activities  Savings of inhabitant’s time “The profound technologies are those which disappear” (Weiser, 1991)

12. SAJAL K. DAS CReWMaN Smart Home Prototypes /Projects  Aware Home (GA-Tech) – Determination of Indoor location and activities  Intelligent Home (Univ. Mass.) – Multi-agent systems technology for designing an intelligent home  Neural Network House (Univ. Colorado, Boulder) – Adaptive control of home environment (heating, lighting, ventilation)  House_n (MIT) – Building trans-generational, interactive, sustainable and adaptive environment to satisfy the needs of people of all age  Easy Living (Microsoft Research) – Computer vision for person-tracking and visual user interaction  Internet Home (CISCO) – Effects of Internet revolution in homes  Connected Family (Verizon) – Smart technologies for home-networking

13. SAJAL K. DAS CReWMaN MAVHome at CSE@UTA  MavHome: Managing an Adaptive Versatile Home  Unique project – focuses on the entire home  Creates an intelligent home that acts as a rational agent  Perceives the state of the home through sensors and acts on the environment through effectors (device controllers).  Optimizes goal functions: Maximize inhabitants’ comfort and productivity, Minimize house operation cost, Maximize security.  Able to reason about and adapt to its inhabitants to accurately route messages and multimedia information. http://ranger.uta.edu/smarthome S. K. Das, et al., “The Role of Prediction Algorithms in the MavHome Smart Home Architecture”, IEEE Wireless Communications, Vol. 9, No. 6, pp. 77– 84, Dec. 2002.

14. SAJAL K. DAS CReWMaN MavHome Vision Face recognition, automated door entry Smart sprinklers Lighting control Door/lock controllers, Surveillance system Robot vacuum cleaner Robot lawnmower Intelligent appliances Climate control Intelligent Entertainment Automated blinds Remote site monitoring and control Assistance for disabilities

15. SAJAL K. DAS CReWMaN MavHome: Bob Scenario  6:45 am: MavHome turns up heat to achieve optimal temperature for waking (learned)  7:00 am: Alarm rings, lights on in bed-room, coffee maker in the kitchen (prediction)  Bob steps into bathroom, turns on light: MavHome records this interaction (learning), displays morning news on bathroom video screen, and turns on shower (proactive)  While Bob shaves, MavHome senses he is 2 lbs overweight, adjusts his menu (reasoning and decision making)  When Bob finishes grooming, bathroom light turns off, kitchen light and menu/schedule display turns on, news program moves to the kitchen screen (follow-me multimedia communication)  At breakfast, Bob notices the floor is dirty, requests janitor robot to clean house (reinforcement learning)  Bob leaves for office, MavHome secures the house and operates lawn sprinklers despite knowing 70% predicted chance of rain (over rule)  In the afternoon, MavHome places grocery order (automation)  When Bob returns, grocery order has arrived and hot tub is ready (just-in-time).

16. SAJAL K. DAS CReWMaN MAVHome: Multi-Disciplinary Research Project  Seamless collection and aggregation (fusion) of sensory data  Active databases and monitoring  Profiling, learning, data mining, automated decision making  Learning and Prediction of inhabitant’s location and activity  Wireless, mobile, and sensor networking  Pervasive computing and communications  Location- and context-aware middleware services  Cooperating agents – MavHome agent design  Multimedia communication for entertainment and security  Robot assistance  Web monitoring and control

17. SAJAL K. DAS CReWMaN MAVHome Agent Architecture  Hierarchy of rational agents to meet inhabitant’s needs and optimize house goals  Four cooperating layers in an agent  Decision Layer Select actions for the agent  Information Layer Gathers, stores, generates knowledge for decision making  Communication Layer Information routing between agents and users/external sources  Physical layer Basic hardware in house House Agent Rooms/ robots Agent Network / mobile network … Agent Network / mobile network … Appliances/ robots Transducers/ actuators User Interface External resources Physical Sensors Actuators Networks Agents Communication Routing Multimedia download Information Data Mining Action Prediction Mobility Prediction Active database Decision MDP/policy Reinforcement learning Multiagent systems/ communication

18. SAJAL K. DAS CReWMaN Indoor Location Management  Location Awareness  Location (current and future) is the most important context in any smart computing paradigm  Why Location Tracking ?  Intelligent triggering of active databases  Efficient operation of automated devices  Guarantees accurate time-frame of service delivery  Supports aggressive teleporting and location-aware multimedia services -- seamless follow of media along inhabitant’s route  Efficient resource usage by devices -- Energy consumption only along predicted locations and routes that the inhabitant is most likely to follow

19. SAJAL K. DAS CReWMaN Location Representation  Location Information  Geometric – Location information in explicit co-ordinates  Symbolic - Topology-relative location representation  Blessings of Symbolic Representation  Universal applicability in location tracking  Easy processing and storage  Development of a predictive framework

20. SAJAL K. DAS CReWMaN Indoor Location Tracking Systems Research Prototypes Underlying Technology Location DataGranularity Active Badge (Univ. of Cambridge) Infrared Symbolic Room-level Active Bats (Univ. of Cambridge) Ultrasonic Geometric 9 cm Cricket (MIT) RF and Ultrasound Symbolic 4 x 4 feet RADAR (Microsoft) IEEE 802.11 WLANs Symbolic 3 – 4.3 m Smart Floor (Georgia Tech) Pressure Sensors GeometricPosition of sensors Easy Living (Microsoft) Vision Triangulation Symbolic variable Motion Star Scene Analysis Geometric 1 m

21. SAJAL K. DAS CReWMaN Inhabitant’s Movement Profile  Efficient Representation of Mobility Profile  In-building movement sampled as collection of sensory information  Symbolic domain helps in efficient representation of sensor-ids  Role of Text Compression  Lempel Ziv type of text compression aids in efficient learning of inhabitant’s mobility profiles (movement patterns)  Captures and processes sampled message in chunks and report in encoded (compressed) form  Idea: Delay the update if current string-segment is already in history (profile) – essentially a prefix matching technique using variable-to-fixed length encoding in a dictionary – minimizes entropy  Probability computation: Prediction by partial match (PPM) style blending method – start from the highest context and escape into lower contexts

22. SAJAL K. DAS CReWMaN MavHome Floor Plan and Mobility Profile  Bob’s movement profile: a j k k o o j h h a a j k k o o j a a j k k o o j a a j k k …  Incremental parsing results in phrases: a, j, k, ko, o, jh, h, aa, jk, koo, ja, aj, kk, oo, jaa, jkk,... Sample Floor-plan Graph-Abstraction  Possible contexts : jk (order-2), j (order-1),  (order-0)

23. SAJAL K. DAS CReWMaN Trie Representation and Phrase Frequencies jk (order-2)j (order-1)  (order-0) k|jk (1)  |jk (1) a|j (1) aa|j (1) k|j (1) kk|j (1) h|j (1)  |j (2) a(4) aa(2) aj(1) j(2) ja(1) jaa(1) jk(1) jh(1) k(4) ko(1) koo(1) kk(2) o(4) oo(2) h(2)   Probability of jaa:  Absence in order-2 and order-1; escape probability in each order: ½  Probability of jaa in order-0: 1/30  Combined probability of phrase jaa : (½) (½ )(1/30) = 0.0048  a (7)j (7)o (6)h (2)k (8) j (1)a (2) k (1)a (1) h (1)a (2)k (2) o (2) o (1) o (2) Phrases and frequencies of different orders Phrases: a, j, k, ko, o, jh, h, aa, jk, koo, ja, aj, kk, oo, jaa, jkk,...

24. SAJAL K. DAS CReWMaN = Probability Computation of Phrases  Probability of k  ½ at the context of order-2  Escaping into next lower order (order-1) with probability: ½  Probability of k at the order-1 (context of “kk”): 1/(1+1) = ½  Probability of escape from order-1 to lowest order (order-0): ½  Probability of k at order-0 (context of  ): 4 / 30  Combined probability of phrase k = ½ + ½ { ½ + ½ (4/30) } = 0.509 jk (order-2)j (order-1)  (order-0) k|jk (1)  |jk (1) a|j (1) aa|j (1) k|j (1) kk|j (1) h|j (1)  |j (2) a(4) aa(2) aj(1) j(2) ja(1) jaa(1) jk(1) jh(1) k(4) ko(1) koo(1) kk(2) o(4) oo(2) h(2) 

25. SAJAL K. DAS CReWMaN Phrase Probabilities 0.0048 0.0905 0.0809 0.0048 ja jaa jk jh a aa aj 0.5905 0.0809 0.0048 0.0195 0.0095 0.0809 0.0095 k kk ko koo o oo h j Phrase Probability Probabilities of individual locations can be estimated by dividing the phrase probabilities into their constituent symbols according to symbol-frequency and adding up all such frequencies for a particular symbol (location) Total probability for location k is: 0.5905 + 0.0809 + 0.0048/2 + 0.0048/3 = 0.6754  Bob’s movement profile: a j k k o o j h h a a j k k o o j a a j k k o o j a a j k k …

26. SAJAL K. DAS CReWMaN Probability Computation of Individual Locations LocationProbability kahojkahoj 0.6754 0.1794 0.0833 0.0346 0.0207  Bob’s movement profile: a j k k o o j h h a a j k k o o j a a j k k o o j a a j k k  Phrases: a, j, k, ko, o, jh, h, aa, jk, koo, ja, aj, kk, oo, jaa, jkk,...  Probabilistic prediction of locations (symbols) based on their ranking  Prime Advantages of Lempel-Ziv type compression – most likely location is predicted  Prediction starts from k and proceeds along a, h, o and j a j h k o

27. SAJAL K. DAS CReWMaN Characterizing Mobility from Information Theory  Movement history: A string “ v 1 v 2 v 3 …” of symbols from alphabet  Inhabitant mobility model: V = {V i }, a (piece-wise) stationary, ergodic stochastic process where V i assumes values v i   Stationarity: {V i } is stationary if any of its subsequence is invariant with respect to shifts in time-axis  Essentially the movement history “ v 1, v 2, …, v n ” reaches the system as C(w 1 ), C(w 2 ), …, C(w n ) where w i s are non-overlapping segments of history v i and C(w i )’s are their encoded forms  Minimizes H(X) and asymptotically outperforms any finite-order Markov model  The number of phrases is bounded by the relation:

28. SAJAL K. DAS CReWMaN Entropy Estimation  Bob’s movement profile: a j k k o o j h h a a j k k o o j a a j k k o o j a a j k k …  For a particular depth d of an LZ trie, let H(V i ) represent entropy at i th level. Running-average of overall entropy is:  a (7)j (7)o (6)h (2)k (8) j (1)a (2) k (1)a (1) h (1)a (2)k (2) o (2) o (1) o (2)

29. SAJAL K. DAS CReWMaN LeZi-Update: Location Prediction Scheme Init dictionary, phrase w loop wait for next symbol v if (w.v in dictionary) w := w.v else encode add w.v to dictionary w := null forever Initialize dictionary := empty loop wait for next codeword decode phrase := dictionary[i].s add phrase to dictionary increment frequency of every prefix of every suffix of phrase forever Encoder Decoder  A paradigm shift from position based update to route based update  Encoder: Collects symbols and stores in the dictionary in a compressed form Decoder: Decodes the encoded symbols and update phrase frequencies

30. SAJAL K. DAS CReWMaN Predictive Framework: Route Tracking  Probability of a set of route sequences depends exponentially on relative entropy between actual route-distribution and its type-class  Route-sequences away from actual distribution have exponentially smaller probabilities  Typical-Set – Set of sequences with very small relative entropy  Small subset of routes having a large probability mass that controls inhabitant’s movement behavior in the long run  Concept of Asymptotic Equipartition Property (AEP) helps capture inhabitant’s typical set of routes

31. SAJAL K. DAS CReWMaN Probability Computation of Typical Routes  From AEP, typical routes classified as: {  : 2 -1.789 L(  ) -   Pr[  ]} where L(  ) is the length of phrase  and  is a very small value  Threshold-probability of inclusion of a phrase into typical-set depends on its length L(  )  At our context: L(  ) Threshold Probability 1 0.289 2 0.080 3 0.002

32. SAJAL K. DAS CReWMaN Capturing Typical Routes 0.0048 0.0905 0.0809 0.0048 ja jaa jk jh a aa aj 0.5905 0.0809 0.0048 0.0195 0.0095 0.0809 0.0095 k kk ko koo o oo h j Phrase Probability  At this point of time and context, the inhabitant is most likely to move around the routes along Bedroom 2, Corridor, Dining room and Living room  Typical Set of route segments comprises of : { k, kk, koo, jaa, aa }

33. SAJAL K. DAS CReWMaN Bob’s Movement along Typical Routes a j k o Typical Route: k o o k j a a Bedroom 2, Corridor, Dining room and Living room

34. SAJAL K. DAS CReWMaN Energy Consumption  Static Energy Plan  Devices remain on from morning until the inhabitant leaves for office and again after return at the end of the day.  Let P i : power of i th device; M : maximum number of devices; t : device-usage time; p(t) : uniform PDF.  Expected average energy consumption:  Using typical values of power, number and usage-time for lights, air-conditioning and devices like television, music-system, coffee- maker from standard home, static energy plan yields ~ 12–13 KWH average daily energy consumption. Worst-Case scenario

35. SAJAL K. DAS CReWMaN Energy Consumption  Optimal (Manual) Energy Plan  Every device turned on and off manually during resident’s entrance and exit in a particular zone.  P i,j : power of i th device in j th zone;  : max # devices in a zone; R : # zones; t : device-usage time in a zone; p(t) : uniform PDF.  Expected average energy consumption:  Using standard power usage, optimal energy plan results in ~ 2–2.5 KWH of average daily energy consumption. Optimal Scenario But lacks automation and needs constant manual intervention

36. SAJAL K. DAS CReWMaN Energy Consumption  Predictive Energy Plan:  Devices turned on and off based on the prediction of resident’s typical routes and locations ( Incorrect prediction incurs overhead)  Devices turned on in advance – existence of time lag (  t ) s : predictive success-rate. As s  1, E[energy predict ]  E[energy opt ]  For the scenario, predictive scheme yields ~3-4 KWH consumption  Successful prediction reduction of manual operations and saving of inhabitant’s invaluable time inhabitant’s comfort

37. SAJAL K. DAS CReWMaN Discrete Event Simulator Simulation Structure  Event types: Daily actions of a user, e.g., sleeping, dining, cooking, etc.  Event Queue  Priority Queue for buffering events  Events ranked according to time stamp.  Event Initializer  Generates the first event and pushes it into the event queue  Event Processing  Carried out with every event  Calls the event generator to generate next event and pushes it into the queue  Calls various action modules depending upon the type of event

38. SAJAL K. DAS CReWMaN Simulation: Assumptions  Simulation Duration: 70 days  Different life-styles at weekdays and weekends  Mobility initiated as the inhabitant wakes up in the morning and starts daily-routine  Inhabitant’s residence-time at every zone – uniformly distributed between a maximum and a minimum value  Negligible delay between sensory data acquisition and actuator activation  Prediction occurs while leaving every zone  In inhabitant’s absence, the house has minimal activity to conserve energy resources

39. SAJAL K. DAS CReWMaN Granularities of Prediction  Predicting next zone  Inhabitant’s immediate next zone / location  A coarse level movement pattern in different locations  Predicting typical routes / paths  Inhabitant’s typical routes along with zones  More granular indicating inhabitant’s movement patterns  Predicting next sensor  Every next sensor predicted from current sensor  Large number of predictions lead to system overhead  Predicting next device  Predict every next device the inhabitant is going to use  Details of inhabitant’s activities can be observed

40. SAJAL K. DAS CReWMaN A Snapshot of Simulation Master bedroom Closet closet Bedroom Restroom Wash room kitchen Living Room kitchen 0 20 10 30 40 50 60 70 80 90 100 Success Rate Corridor kitchen Dining Room 4 2 6 8 10 12 14 Energy Savings Static Optimal Predicted ActualCorrect Prediction Dining Room kitchen Garage

41. SAJAL K. DAS CReWMaN Learning Curve and Predictive Accuracy  85% – 90% accuracy in predicting next sensor, zone and typical route  Route prediction accuracy slightly lower than location prediction, yet provides more fine-grained view about inhabitant’s movements  Only 4-5 days to be cognizant of inhabitant’s life-style and movements  Higher granularity keeps device prediction accuracy low (63%)

42. SAJAL K. DAS CReWMaN Memory Requirements Variation of Success-rate with table-size  85% success rate with only 3–4 KB memory for inhabitant’s profile  Small size typical set (5.5% -- 11% of total routes) as typical routes

43. SAJAL K. DAS CReWMaN Energy Savings Reduction in Average Energy Consumption  Energy along predicted routes / locations only – minimum wastage  Average energy consumption – 1.4 * (optimal / manual energy plan)  65% – 72% energy savings in comparison with current homes

44. SAJAL K. DAS CReWMaN Reduction in Manual Operations  Prediction accuracy  reduction of manual operations of devices  brings comfort and productivity, saves time  80% – 85% reduction in manual switching operations

45. SAJAL K. DAS CReWMaN Future Work  Route prediction and resource management in multi-inhabitant (possibly cooperative) homes  Design and analysis of location-aware wireless multimedia communication in smart homes  Integration of smart homes with wide area cellular networks (3G wireless) for complete mobility management solution  QoS routing in resource-poor wireless and sensor networks  Security and privacy issues

46. SAJAL K. DAS CReWMaN  A. Roy, S. K. Das Bhaumik, A. Bhattacharya, K. Basu, D. Cook and S. K. Das, “Location Aware Resource Management in Smart Homes”, Proc. of IEEE Int’l Conf. on Pervasive Computing (PerCom), pp. 481-488, Mar 2003.  S. K. Das, D. J. Cook, A. Bhattacharya, E. Hierman, and T. Z. Lin, “The Role of Prediction Algorithms in the MavHome Smart Home Architecture”, IEEE Wireless Communications, Vol. 9, No. 6, pp. 77– 84, Dec. 2002.  A. Bhattacharya and S. K. Das, “LeZi-Update: An Information Theoretic Framework for Personal Mobility Tracking in PCS Networks”, ACM Journal on Wireless Networks, Vol. 8, No. 3, pp. 121-135, Mar-May 2002.  A. Bhattacharya and S. K. Das, “LeZi-Update: An Information Theoretic Approach to Track the Mobile Users in PCS Networks”, Proc. ACM Int’l. Conference on Mobile Computing and Networking (MobiCom’99), pp. 1-12, Aug 1999 (Best Paper Award). Selected References

47. SAJAL K. DAS CReWMaN  D. J. Cook and S. K. Das, Smart Environments: Algorithms, Protocols and Applications, John Wiley, to appear, 2004.  A. Bhattacharya, “A Predictive Framework for Personal Mobility Management in Wireless Infrastructure Networks”, Ph.D. Dissertation, CSE Dept, UTA (Best PhD Dissertation Award), May 2002.  A. Roy, “Location Aware Resource Optimization in Smart Homes”, MS Thesis, CSE Dept, UTA (Best MS Thesis Award), Aug 2002.  S. K. Das, A. Bhattacharya, A. Roy and A. Misra, “Managing Location in ‘Universal’ Location-Aware Computing”, in Handbook in Wireless Networks (Eds, B. Furht and M. Illyas), Chapter 17, CRC Press, June 2003. Selected References

48. SAJAL K. DAS CReWMaN Technology Forecasts (?) ‘ Heavier-than air flying machines are not possible’ Lord Kelvin, 1895 ‘I think there is a world market for maybe five computers’ IBM Chairman Thomas Watson, 1943 ‘640,000 bytes of memory ought to be enough for anybody’ Bill Gates, 1981 ‘The Internet will catastrophically collapse in 1996’ Robert Metcalfe ‘Long before the year 2000, the entire antiquated structure of college degrees, majors and credits will be a shambles’ Alvin Toffler

49. SAJAL K. DAS CReWMaN Concluding Remarks “A teacher can never truly teach unless he is still learning himself. A lamp can never light another lamp unless it continues to burn its own flame. The teacher who has come to the end of his subject, who has no living traffic with his knowledge but merely repeats his lesson to his students, can only load their minds, he cannot quicken them”. Rabindranath Tagore (Nobel Laureate, 1913) Rabindranath Tagore (Nobel Laureate, 1913)