Mining Minds Mr. Amjad UsmanMr. Amjad Usman19-July-2014KHU High-level Context Awareness

/ Presentation Outline Introduction Motivation Related Work Limitations Proposed Architecture Tools and Technologies Development Timeline Current Status 2

/ Introduction According to Dey, Abowd and Salber (2001) Context is any information that can be used to characterize the situation of entities (i.e. whether a person, place or subject) that are considered relevant to the interaction between a user and an application, including the user and the application themselves Means Context is very dynamic and transient Low-level Context is the raw data coming from sensors / source High-level Context is the abstract information extracted from logically relevant low- level context 3

/ Motivation End-users are interested in high-level context, not raw data For example “It’s raining” (high-level) is preferred rather than “humidity: 77%“ (low-level) To discover Higher-level Context Information Context fusion is required Find the semantic relationship By integrating all the information relevant to one’s life It can give us a real sense of Life logging 4

/ Motivation 5 Context Information Sensor data from sensors Activity data from smartphones Nutrition data Social media data Recommending Services Lifestyle & Lifecare Exercise and nutrition Social interaction Behavior analysis & prediction Behavior Modeling Lifestyle Analysis, Prediction, and Recommendations Context Information

/ User Profile Social Media Life log Activity Enviro nment Diet Proposed Idea Integration Social Media Information Diet Information Activity Information Profile Information Environment Information 6 Behavior Modeling Activity Context Information  Independent logs Life log Information  Semantically related

/ Related Work Ontology/ Subdomain CoBrA-Ont [2003] CoDAMoS [2004] SOUPA [2005] Delivery Context [2009] DeviceXXX EnvironmentXXX LocationXXXX NetworkX RoleXX TimeXXX UserXXX Implementation Available XXX 7

/ Related Work Ontology/ Subdomain CONON [2003] Situation Ont [2004] mIO [2005] PiVOn [2009] PalSPOT [2011] DeviceXXXXX EnvironmentXXXXX LocationXXXXX NetworkXX RoleXX TimeXXXXX UserXXXXX Implementation Available XX 8

/ Limitations Domain Specific Context Models Information exist in different types of Logs but are stand alone No integrated system Very few systems consider social interaction None of the existing systems use any behavioral model High level contexts are extracted using context interpretation and aggregation techniques regardless of Context verification, validation and fusion 9

/ Proposed Architecture Long/Short-term Behavior Analysis Context Awareness Mapper and Transformer Context Interpreter Context Analyzer Rule base Parser Decision Making DTD2OWL OWL Ontology XML2OWL Query Generator Activity Retrieval Match Making Rule-based Filtering Decision Propagation Situation Analyzer Prediction and Reasoner Recommendation Manager Low Level Context-awareness HDFS Data Interface Intermediate Data Life log Modeling Context Receiver Context Representation Context Verification Context Fusion Context Logging Behavior Modeling Parser Life Log Repository 10 Feature Selection Pattern Classification Pattern Identification Life log Extractor

/ Scenario Context Awareness | Long/Short-term Behavior Analysis Prediction and Reasoner Recommendation Manager Low Level context- awareness HDFS Data Interface Intermediate Data Long/short term Behavior Analysis Mapper and Transformer Context Interpreter Context Analyzer Parser Decision Making DTD2OWL OWL Ontology Query Generator Activity Retrieval Match Making Rule-based Filtering Decision Propagation Situation Analyzer Rule base 5 7 Activities detected: walking Location detected: Kitchen Time noticed: 12:00:00 Ali Exercising Sentiment: tired Rule 1 (activity=eating AND time<8:00:00)  Taking Breakfast Rule 2 (activity=eating AND time=12:00:00)  Taking Lunch Rule 3 (activity=eating AND time=18:00:00)  Taking Dinner eating Kitchen 12:00:00 Ali "SELECT ?activityName ?hasConsequentAction ?type ?performedBy ?time WHERE { ?activityName." + " };  In Kitchen  Eating  12:00:00  Taking Lunch in Kitchen Rule 2 Matched (activity=eating AND time=12:00:00)  Taking Lunch XML2OWL 11

/ Scenario Context Awareness | Long/Short-term Behavior Analysis Context Receiver Context Representation Context Fusion Context Verification and Logging Life Log Repository Behavior Analysis Life Log Extractor Parser Context ConverterContext RepresentationContext Mapper Horizontal Context Fusion Vertical Context Fusion Data Extractor Data Logger Query Formulation Data Fetcher Data Processing Pattern Classification Pattern Identification Feature Selection Consistency Verification Semantic Structural Log Context Existence Verification Low Level context-awareness High Level Context-awareness 11 HDFS Data Interface Intermediate Data 1 Prediction and Reasoner Recommendation Manager Verified Data eating Kitchen 12:30:0 0 Ali Lunch, Exercising Complete Information of User like profile, activities performed when and where, tweets, etc Behavior Analysis Lunch No proper timing Exercising Regular High Level Context: Exercising Structured Data Ali,Lunch,Hotel, , 12:00:00 walking, sitting, eating, tweet: tired Activity detected: walking, eating Location detected: Kitchen Time: 12:30:00 12

/ Tools and Technologies Knowledge Representation ––RDF / RDFS / OWL Ontologies ––Protégé as IDE Programming Language & API ––Java ––Jena, Twitter Querying Language ––SPARQL Reasoner ––Racer Pro / Pellet / FaCt++ 13

/ Development Time Line Four Year Work Break Down 14 Literature Study: Context Modeling Analytical Study: Context Conversion Analytical Study: Context Interpretation Literature Study: Life log System Context logging in Life log Context Integration & Verification Literature Study: Behavior Representation Study: Behavior Modeling Algorithmic Study: Behavior Analysis System Development Unit Testing Integration Testing Context Model Context Mapping Technique Context Analyzer Life log Model Life log Parser Fusion + Verification Techniques Features Selection Behavioral Model Creation Model Selection Pattern Identification Algorithm: Long-term / Short-term Analysis High-level Context Awareness Test Report First Year Second Year Third Year Fourth Year Context awareness Life log Management Behavioral Analysis Documentation Prototype Development Technical Guide

/ Current Status Context Awareness ––Literature Survey regarding ––Context Modeling Techniques ––Context Conversion & Mapping Algorithms Long/Short-term Behavior Analysis ––Literature Survey regarding Lifelog Design and Development LifeLog Model: OWL+Dynamic/Static Storage: Ont-RDB / JenaTDB / RDF Nature & type of Logs 15

/ Context Modeling Work Flow 16

/ What to store in Life log? 17

/ Behavior Analysis & Prediction Work Flow 18

Questions Thank You! address: address: Cell Number: Cell Number:

/ Long/Short-term Behavior Analysis Deployment Diagram 20

/ Proposed Architecture Prediction and Reasoner Recommendation Manager Low Level context-awareness HDFS Data Interface Intermediate Data Long/Short-term Behavior Analysis Context Awareness 21

/ Proposed Architecture Prediction and Reasoner Recommendation Manager Low Level context-awareness HDFS Data Interface Intermediate Data Long/Short-term Behavior Analysis Mapper and Transformer Context Interpreter Context Analyzer Rule base Parser Decision Making DTD2OWL OWL Ontology XML2OWL Query Generator Activity Retrieval Match Making Rule-based Filtering Decision Propagation Situation Analyzer Context Awareness | Long/Short-term Behavior Analysis 22

/ Proposed Architecture Low Level context-awareness HDFS Data Interface Intermediate Data Context-Awareness Context Receiver Context Representation Context Fusion (Vertical / Horizontal) Context Verification and Logging Life Log Repository Behavior Analysis Life Log Extractor Parser Prediction and Reasoner Recommendation Manager Context Awareness | Long/Short-term Behavior Analysis 23

/ Proposed Architecture Long/Short-term Behavior Analysis Context Awareness Mapper and Transformer Context Interpreter Context Analyzer Rule base Parser Decision Making DTD2OWL OWL Ontology XML2OWL Query Generator Activity Retrieval Match Making Rule-based Filtering Decision Propagation Situation Analyzer Prediction and Reasoner Recommendation Manager Low Level Context-awareness HDFS Data Interface Intermediate Data Lifelog Modeling Context Receiver Context Representation Context Verification Context Fusion Context Logging Behavior Modeling Parser Life Log Repository Life Log Extractor Behavior Checker Model Creater Behavior Descriptor Model Validator Behavior Analyzer 24

/ Motivation To integrate the different context information emerging from diverse sources to identify user’s behavior in order to analyze the user’s lifestyle and provide recommendations to promote active lifestyle Social Media Information Diet Information Activity Information Profile Information Environment Information Long / Short-term Behavior Modeling Life log 25

/ Motivation Integration Social Media Information Diet Information Activity Information Profile Information Environment Information 26 Social Media Information Diet Information Activity Information Environment Information Profile Information Short-term Behavior Modeling Long-term Behavior Modeling