Amel Bouzeghoub Serge Garlatti Context Modeling & Adaptation Amel Bouzeghoub Serge Garlatti

Outline Context & Context-Awareness General Model of Context-Awareness Context Modeling Applications Context-Aware Learning Systems Adaptation: Main issues Adaptation Process and Principles Mobilearn Project P-LearNet Project Conclusion 4th-9th july 2009

Is all this information measurable? Defining Context Location, identities of nearby people and objects [Schilit and Theimer] Location, identities of the people around the user, the time of day, season, temperature, etc. [Brown et al.] The user’s location, environment, identity and time [Ryan et al.] The user’s emotional state, focus of attention, location and orientation, date and time, objects, and people in the user’s environment [Dey] The elements of the user’s environment that the user’s computer knows about [Brown] The situation of the user [Franklin & Flaschbart] Is all this information necessary? Is all this information measurable? 4th-9th july 2009

Defining Context (cont’d) Dey defines the context as “any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves.” Schilit et al. claim that the important aspects of context are: where you are, who you are with, and what resources are nearby. They define context to be the constantly changing execution environment Computing environment available processors, devices accessible for user input and display, network capacity, connectivity, and costs of computing User environment location, collection of nearby people, and social situation Physical environment lighting and noise level Which aspects of all situations are important ? 4th-9th july 2009

Defining Context (cont’d) Context is the set of environmental states and settings that either determines an application’s behavior or in which an application event occurs and is interesting to the user [Chen]: Active context: influences the behavior of the application an application automatically adapts its behavior to discovered context Passive context: context that is relevant but not critical an application presents the new or updated context to an interested user or makes the context persistent for the user to retrieve later 4th-9th july 2009

Defining Context (cont’d) Sensor data Other classifications of context: Low-level vs High-level context External (physical) Context that can be measured by hardware sensors location, light, sound, movement, touch, temperature, air pressure, etc. Internal (logical) Mostly specified by the user or captured monitoring the user’s interaction the user’s goal, tasks, work context, business processes, the use’s emotional state, etc. Low-level context motivational social user location computing temporal high-level context Context processing active context passive context Context-aware application 4th-9th july 2009 6

Defining Context (cont’d) 3 entities Places (rooms, buildings, etc.) People (individuals, groups) Things (physical objects, computer components, etc.) Attributes Identity (each entity has an unique identifier) Location (an entity’s position, co-location, proximity, etc.) Status (or activity) Time (used for timestamps to accurately define situation, ordering events, etc.) … 4th-9th july 2009 7

Challenges [Roy Campbell] Context is most useful in dynamic, mobile environments. But what is the relevant information in various situations? Mobility results in continuous updates of context information. How can we efficiently manage this? How can we share context? How do we handle uncertainty of context information? How do we ensure privacy control and management of context information? How do we reach a common understanding of implications and semantics of (shared) context information? Resource restrictions 8 4th-9th july 2009 8

Context-Awareness “A system is context-aware if it uses context to provide relevant information and/or services to the user, where relevancy depends on the user’s task.” [Dey’s definition] 4th-9th july 2009

General Model of Context-Awareness 4th-9th july 2009

Abstract Layer Architecture Sensing: numeric observables Perception: symbolic observables Situation and context Applications Privacy/ Security/ Trust History Discovery/Recovery Usage Generation sensors [J. Coutaz, et al.] 4th-9th july 2009 11

Abstract Layer Architecture (cont’d) The sensing layer generates numeric observables to determine meaning from numeric observables, the system must perform transformations Physical sensors RFID, camera, microphone, accelerometer, GPS, thermometer Virtual sensors From software: browsing an electronic calendar, a travel booking system, emails, mouse movements, keyboard input Logical sensors Combination of physical and virtual sensors The perception layer is independent of the sensing technology and provides symbolic observables at the appropriate level of abstraction Changes of Context !! Most of contextual information changes as time goes on Context changes can be sensed by, periodic polling, advertising, selective polling 4th-9th july 2009 12

Abstract Layer Architecture (cont’d) The situation and context layer identifies the current situation and context from observables The exploitation layer acts as an adapter between the application and the infrastructure. This is where applications express their requests for context services at a high level of abstraction At every level of abstraction, one must incorporate mechanisms and facilities to support privacy, trust, and security as well as history management and discovery/recovery 4th-9th july 2009 13

Context Modeling 4th-9th july 2009

Motivations Context modeling requirements: Formality Support for efficient reasoning Support for imperfect context information (imprecise, ambiguous, incomplete information) Appropriate abstractions to support requirements analysis, design and programming tasks Support for interoperability 4th-9th july 2009

Context Modeling Context Model Define and store context data in a machine processable form Context Modeling Approaches Key-Value Models Markup Scheme Models Graphical Models Object Oriented Models Logic Based Models Ontology Based Models 4th-9th july 2009

Context Modeling Approaches Key-Value-Pairs Models [Context Toolkit] Most simple category of models Not very efficient for more sophisticated structuring purposes Only exact matching (set loc=room_134) No inheritance Standard Retrieval Method: Linear Search Markup Scheme Models [Stick-e] XML-based models Composite Capabilities/Preference Profile (CC/PP), [W3C, 2004], User Agent Profile (UAProf) encoded in RDF/S [Wapforum, 2001] Standard Retrieval Method: Markup Query Language 4th-9th july 2009

Context Modeling Approaches (cont’d) Graphical Models [ORM] UML-like languages (CML) Particularly useful for structuring, but usually not used on instance level Standard Retrieval Method: Transformation Object Oriented Models [Hydrogen] Intention behind object orientation is (as always) encapsulation and reusability Standard Retrieval Method: Algorithm 4th-9th july 2009

Context Modeling Approaches (cont’d) Logic Based Models [eWallet – myCampus] Logic defines conditions on which a concluding expression or fact may be derived from a set of other expressions or facts (reasoning) → context is defined as facts, expressions and rules High degree of formality Cannot inherently deal with uncertain or conflicting information Suit better in cases, where we are certain about the quality of the collected data Standard Retrieval Method: Inference 4th-9th july 2009

Context Modeling Approaches (cont’d) Ontology Based Models [SOCAM, CoBrA, Context Managing Framework] Ontology used as explicit specification of a shared conceptualization Context is modeled as concept and facts Advantages: formal models that can be shared reused, extended combined Standard Retrieval Method: Reasoning 4th-9th july 2009

Summary Approach Requirements dc pv qua inc for app Key-value models - dc: distributed composition pv: partial validation qua: quality of information Inc: incompleteness and ambiguity app: applicability to existing env. for: level of formality Approach Requirements dc pv qua inc for app Key-value models - + Markup scheme models ++ Graphical models OO models Logic based models Ontology based models 4th-9th july 2009

Applications 4th-9th july 2009

Applications - Efficient UI [Team Edward : Hee-Jung Yang and Hyeong-Ho Yoo (2005)] More efficient UI as context changes Brighter display in dark place Simplified text when I am moving normal dark place & user is moving 4th-9th july 2009

Applications - Efficient UI (cont’d) Show only important part of clip for poor network bandwidth or small display Different menu for different place Sort people list by distance or show on a map big display small display Depend on user’s preference 4th-9th july 2009

Applications - Reconfiguration Optimize internal components Choose the fastest connection to server Print with the closest printer 4th-9th july 2009

Applications - Triggering Trigger useful information relevant to current context Car navigation system Explanation about the place where user travels Show cooking guide in kitchen, show work files in office Reminder for specified context When I next meet Serge, tell him ... When next at a market, buy ... 4th-9th july 2009

Applications – Memory Retrieval Save contexts of each event and retrieve them later Who were the girls with Serge at the last meeting in Paris? What was the note I made at a meeting last June in this conference room? I remember Bill and John were there. 4th-9th july 2009

Applications – Memory Retrieval (cont’d) Object is defined by relative context Filename is the context in which it is made We don’t have to remember the filename Interrupted task is resumed if the context revived 4th-9th july 2009

Applications – Sharing Experience Refer other people's contexts Travelers download my contexts of my trip Broadcast my context I send to neighbors the information about cheap theatres 4th-9th july 2009

Context-Aware Learning Systems 4th-9th july 2009

Other definitions… Ogata, & Yano : “ Ubiquitous learning is characterized by providing intuitive ways for identifying right collaborators, right contents and right services in the right place at the right time based on learners surrouding context » Derntl & Hummel: « learning context is used to describe the current situation of a person related to a learning activity. In addition to attributes relying on the physical world model » A. Kurti & M. Milrad: « information and content in use to support a specific activity (being individual or collaborative) in a particular physical environment ». Adaptation Learner-centered + Environmental factors (e.g. location, time of the day, delivery media) Goal: capturing and usage of contextual information in context-aware learning environments 4th-9th july 2009

Research Questions How to model mobile learning context considering individual tasks, community activities, and current environment? How to use context in a particular mobile learning scenario in order to provide adaptation to users? How to develop a flexible system architecture for adaptive activity-based vs situation-based learning? 4th-9th july 2009

Adaptation 4th-9th july 2009

Main Issues Context-Awareness and Adaptation At different levels in the global architecture HCI components Educational and semantic components Middleware components Network components On Mobile Learning

Main Issues Context properties Dynamic “Context-as-construct”. Context settings could be “unpredictable” because it is dynamically created and modified through continual interactions with users and environment. On Mobile Learning

Main issues Context-Awareness and Adaptation Select dynamically the relevant resources Teachers, Learners, Communities, HCI components, Scenarios, activities, multimedia documents Middleware components, web services Network components, etc. According to the current situation As a subset of the context model On Mobile Learning

Adaptation Process and Principles Context-Awareness and Adaptation Resource model assumptions A set of structured features from « External » resources A metadata schema having LOM as a subset: General, technical, Pedagogical, classification, Meta metadata, rights, Version, etc. « Internal » resources A set of features describing a learning object On Mobile Learning

Adaptation Process and Principles Context-Awareness and Adaptation Context model assumptions: a set of structured features scenario, activity, Learner, teacher, Community, Location, time, network, Device, Tools, etc. On Mobile Learning

Adaptation Process and Principles Context and Adaptation Mangement Process On Mobile Learning

Adaptation Process and Principles 1) Resource Retrieval Resources Multimedia documents: images, movies, texts, sounds, etc. Activities, scenarios, Web services, Web page patterns, HCI components, middleware components, etc. Query  Result: a set of resources On Mobile Learning

Adaptation Process and Principles 2) Resource retrieval refinement Filtering process Matching context model against resource metadata or indexing Outcome, at least, two classes: « Good » and « Bad » Or up to 5 classes (ressource clustering) Mutually exclusive equivalence classes!!!!!!! 3) Applying and adaptation method Annotation, hiding, sorting or ranking, etc. On Mobile Learning

according to resource category! Adaptation Process and Principles according to resource category! On Mobile Learning

Adaptation Process and Principles Unfortunately Context settings could be “unpredictable” Not really possible to deal with! Consequences Membership rules combine contextual and metadata properties to classify resources What happens when some properties are not available? Impossible to compute membership? On Mobile Learning

Adaptation Process and Principles How to manage context features to have flexibility as much as possible? Two Case Studies Mobilearn P-LearNet On Mobile Learning

Mobilearn Project

Mobilearn Project Goals Produce an integrated architecture for learners with mobile devices Support for collaborative learning Adaptive human interface Context-aware presentation of content, options and services to support learning activities On Mobile Learning

Mobilearn Project Context Hierarchy Scenarios Determine the relevant context elements To match the context-awareness of the learner’s needs Context What's going over time Context State Elements (currently present) from the Learning and Setting at one particular point in time, space, or goal sequence Context Substate Elements from the Learner and Setting that are relevant to the current focus of learning and desired level of context awareness Context Feature Context Feature Context Feature Context Feature On Mobile Learning

Mobilearn Project Context awareness architecture On Mobile Learning

Mobilearn Project Basis Cycle of operation 1) Gathering and input of context metadata 2) Construction of context substate 3) Exclusion of unsuitable content 4) Ranking of remaining content 5) Output of ranked list of content On Mobile Learning

Mobilearn Project Context feature belong to two types Excluders Items of content, deemed entirely inapropriate for the current context are excluded Belong to class « Bad » and are hidden Or Rankers Increments the score of each content item that has metadata matching the value of any particular context feature The size of increment depends on the salience value associated to the considered context feature On Mobile Learning

Mobilearn Project On Mobile Learning

Mobilearn Project Context awareness and adaptation As the adaptation process is dedicated to a single feature, it is possible to deal with almost « unpredictable » context settings. On Mobile Learning

P-LearNet Project On Mobile Learning

P-LearNet and e-retail requirements What activities I can do in the selling situation? Need to model activities (scenario) How can I get supports or helps? How can I realize an activity when the working situation changes? -Need to integrate learning and working activities -Continuous professional learning at the workplace or elsewhere What and how can I learn during working activities? Seller Need to achieve an activity in different contexts, across different contexts Increasing the system capability to get information about physical and social selling environment to enrich learning at the workplace to improve the market

Context modeling A context management model A context model, A context metadata schema A set of situations A situation: a partial instantiation of the context model consisting of the available features Organized in historical dependencies On Mobile Learning

Goals Designing a context-aware and adaptive mechanism to select relevant resources to achieve activities Finding relevant resources according to the current situation Modeling integrated context-aware learning and working scenarios in the workplace page 56

Two Adaptation Strategies Situation-based Learning Passive Adaptation Proactive Adaptation Recommendation system Depending on learner situation proposes right contents, right services, right activities, right collaborators in the right place at the right time Activity-based Learning Task/method paradigm On Mobile Learning

Situation-based Learning User current situation at in nearBy hasSituation uses in do(role) collaborate nearBy 59 User Location Activity Device Environment Time On Mobile Learning 59

Situation-based Learning Passive adaptation Reduces the number of resources according to user current situation Uses filters: user knowledge, profile and preferences Proactive adaptation Allows an active recommendation of resources without user intervention simply by detecting predefined events or system events Uses ECA rules (Event/Condition/Action). Authors describe resource behaviour depending on events that occur during the situation Prototypical situations, hierarchical organization Authors have to provide pedagogical rules for the course or to precise which rules to use among the set of existing rules. Example: in such situation, (situation in which the learner encounters difficulties and needs help), the author may indicate which action is needed to help the learner: provide an easier or more detailed resource On Mobile Learning

Situation-based Learning: Passive Adaptation Resource Retrieval and Adaptation Input content : Resources Context Metadata Current situation Evaluation Filtering Knowledge Preferences Relevant content Ranking / Annotation Current Situation Passive Adaptation User Queries Weighted content On Mobile Learning

Situation-based Learning: Proactive Adaptation Provides the relevant esources according to current situation Events: Matching Prototypical situations Evaluation Filtering Ranking Annotation Matched situations Current situation Context Metadata Weighted content Relevant , Passive Adaptation Proactive Adaptation ECA / Situational rules Pedagogical rules Searching Resource Resources On Mobile Learning

Activity-based Learning Working and learning activities are intertwined in the workplace How to model integrated learning and working activities in workplace – e-retail system? Seamless Learning and Working Achieve activities according to the current situation New situation  New way to achieve an activity? Scenarios for modeling activities in context and across contexts On Mobile Learning

Activity-based Learning Context-aware scenarios Context-aware learning scenarios at workplace is represented by a hierarchical task model The task/method paradigm Tasks: Represent activities and sub-activities An abstract task is composed of sub-tasks An atomic task can be achieved by a simple procedure and can not be composed of sub-tasks On Mobile Learning

Activity-based Learning Context-aware scenarios Methods: represent a way to achieve a given task according to a context Methods described by contextual features (Web) Services/ resources described by metadata realized by Resource requirement Method1 Retrieval-Matching Task (Web) Service requirement Method2 Method3 function specification SubTask1 Method4 Control structure SubTask2 Define a decomposition of a task into subtasks On Mobile Learning

Activity-based Learning On Mobile Learning

Activity-based Learning Select the relevant methods to achieve a given task according to the current situation On Mobile Learning

Activity-based Learning Service retrieval and adaptation On Mobile Learning Page 16

Conclusion Some Open Issues How to manage context features and adaptation process How to manage adaptation at all levels? HCI components Educational and semantic components Middleware components Network components On Mobile Learning