1. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Ubiquitous User Modeling Prüfungsausschuss: Vorsitzender: Prof. Dr. Thorsten Herfet Berichterstatter: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster Berichterstatter: Prof. Dr. Jon Oberlander (University of Edinburgh) Akademischer Mitarbeiter der Fakultät: Dr. Jörg Baus Promotionskolloquium Dominik Heckmann Naturwissenschaftlich-Technische Fakultät I der Universität des Saarlandes Fachbereich 6.2 Informatik 7. Dezember 2005 14:00s.t. 1. Motivation & Background 2. Suggested Solutions (KR) 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook 1

2. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Preface As we walk, our locomotion reveals our destinations. As we talk, our speech reveals our intentions. As we gesture, our motions reveal our thoughts. As we read, our gaze reveals our focus of attention. As we type, our keystrokes reveal our intentions. As we surf the web, our clicks reveal our interests. Jon Orwant, DOPPELGÄNGER Project [Orwant, 1995] 1. Motivation & Background 2

3. 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook From Empathy to User Modeling A A B B Human-Human Interaction B B A A Systems adapt to their users: User-Adaptivity Systems model their users: User Modeling people model their interaction partner age, gender, mother language cognitive load, time pressure, emotion mood, clothes, plans, knowledge people show empathic behavior We adapt our vocabulary (age) We adapt our speech volume (hearing impaired) We adapt our speech rate (time pressure) A A Human-Computer Interaction A A User Model (aspects of the user) User Model (aspects of the user) Upward Inference (model acquisition) Upward Inference (model acquisition) Input Information (about the user) Input Information (about the user) Downward Inference (model application) Downward Inference (model application) Output Hypothesis (to assist the user) Output Hypothesis (to assist the user) [Jameson,2001] 1. Motivation & Background 3

4. 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook From User Modeling to Ubiquitous User Modeling More and more interactions take place between humans and different stationary, mobile or web-connected IT-systems in daily life. And there is a shift from the well understood desktop metaphor to the metaphors of mobile computing, ubiquitous computing [Weiser,1991] and intelligent environments If we manage to integrated all distributed, user-related assumptions (that are currently applied by these systems individually) into one consistent model, then we could expect several improvements We expect that ongoing evaluation of user behavior with systems that share their user models will improve the coverage, the level of detail, and the reliability of the integrated user models (and thus allow better functions of adaptation) [Weiser,1991] 1. Motivation & Background 4

5. 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook OfficeAirportHotel Variety of Environments To illustrate Ubiquitous User Modeling within the SFB Airport Scenario Pedestrian Navigation Pedestrian Navigation Shopping Guide Shopping Guide Restaurant Guide Restaurant Guide Adaptive Hypertext Adaptive Hypertext Variety of Applications Adaptive (Airport) Navigation Adaptive (Airport) Navigation Adaptive Dialogue Interaction Adaptive Dialogue Interaction Product Recommen- dation Product Recommen- dation Adaptive Web-Sites Adaptive Web-Sites Location-based Information Presentation Location-based Information Presentation Variety of Adaptation Functionality TV WWW Info Kiosk Info Kiosk Terminal Hall Terminal Hall Gate Airplane Shop WWW Variety of Locations Rest Room Rest Room Variety of Situations [images from the collaborative research center on resource-adaptive cognitive processes] 1. Motivation & Background 5

6. 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook How do we define Ubiquitous User Modeling? Definition: Ubiquitous user modeling describes ongoing modeling and exploitation of user behavior with a variety of systems that share their user models Conceptual View: Modeling & Exploitation User Behaviour, User State, User Context, User Interests, User Knowledge, User Plans Exploitation User ? ? Sharing ? ? IT System with User Modeling Component IT System with User Modeling Component IT System with User Modeling Component IT System with User Modeling Component IT System with User Modeling Component IT System with User Modeling Component 1.2.3. Modeling User 1. Motivation & Background with the initial idea of exchanging partial user models 6  Related Work

7. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Related Work: Selected (non-ubiquitous) Generic User Modeling Systems Doppelgänger [Orwant,1994] –produced customized newspapers on the basis of user preferences –applied a centralized user model in LISP-like lists –communication between user and server by email –outdated, but it envisioned already the idea of ubiquitous user modeling UM Toolkit [Kay,1995] –first user modeling server with the goal of making the user model itself accessible –accretion user model = uninterpreted collection of evidence for each component –interpretation of the evidences at runtime by the application Personis [Kay,Kummerfeld,Lauder,2002] –support user scrutiny and user control –so-called views are shared between the server and each application –only authorized systems can contribute to the user model Deep Map User Modeling System [Fink,Kobsa,2002] –system for personalized city tours –user‘s interests and preferences in the travel and tourism domain –allows a distributed use of a centralized user model 1. Motivation & Background 7  Differences

8. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Differences, Implications, Tasks and Design Decisions Differences to generic user modeling systems: –Ubiquitous user modeling can be differentiated by the three additional concepts: ongoing modeling, ongoing sharing and ongoing exploitation Direct implications: –Ubiquitous user modeling implies that the user’s behavior and the user’s state could constantly be tracked at any time, at any location and in any interaction context, which leads to an extended need for privacy and user control Main identified Tasks: –enable user model exchange and knowledge-sharing between user-adaptive systems on the web and within instrumented environments –enable facilities for the user to inspect and control the represented and exchanged user-related data Main Design Decisions: –to support decentralization, allow inconsistencies, use conflict resolution –to support scrutability, modularity, keep clearity, integrate external ontologies 1. Motivation & Background 8

9. 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Selected Research Questions with an Overview of suggested Solutions 1.How can a user model exchange language be designed for the communication between different user modeling applications? –SituationalStatements = relation-based user model&context representations –UserML & UserQL = RDF-based user model exchange languages 2.How can conceptual knowledge about user model dimensions be collected and organized for a semantic web ontology? –G UMO = OWL-based ontology for user models –UbisOntology = OWL-based ontology for ubiquitous computing 3.How can users inspect and control their distributed user models? –UserModelEditor = scrutiny interface, online –PrivacyEditor = integrated editor, based on P3P 4.How does a reasonable architecture for decenralized user modeling look? –U2M UserModelService = modularized, webservice-based, distributed 2. Suggested Solutions 1. Motivation & Background 9 2. 4. 7. 11.

10. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook administration privacy explanation What will be exchanged? Mainpart + Meta Data = „Peter ist under high time pressure“ Which meta data is important for ubiquitous user modeling? When and where is the statement valid? Who claims this and which explanation is given? What is the evidence and the confidence? What will be done with my user model? When will this information be deleted? Who is the owner of this information? What are the privacy settings? How can the statement be uniquely identified? Can the statement be grouped with others? situation mainpart SituationalStatement 2. Suggested Solutions 10

11. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook administration privacy explanation situation mainpart From Statements to UserML Subject = Peter Auxiliary = hasProperty Predicate = timePressure Range = low-medium-high Object = high Start = 2005-04-16T19:15 End = 2005-04-16T19:25 Durability = few minutes Location = airport.dutyfree Position = x34-y22-z15 Key = ******** Owner = Peter Access = friends-only Purpose = research Retention = few days Mainpart Situation Privacy Situational Statement / Box Source = peter.repository Creator = airport.inference Method = deduction13 Evidence = id2, id3 Confidence = most-probably Explanation id = 23 unique = u2m.org#154123 replaces = u2m.org#154006 group = UserModel notes = ;-( Administration 2. Suggested Solutions UserML / RDF-XML <rdf:RDF xmlns:rdf=“http://www.w3.org/1999/02/22-rdf-syntax#“ xmlns:st=“http://www.u2m.org/2003/situation#“ xml:base=“http://www.u2m.org/2003/statements“> A1 A2 A3 A4 A5 A6 A7 A8 A9 A10 A11 A12 A13 A14 A15 A16 A17 A18 A19 A20 A21 A22 A23 A24 A25 11

12. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Reification-based RDF Representation for UserML UserML / RDF-Graph (Reification) M1 P3P2P1 M5 S3 S2S1 M2/M3/M4 Mainpart (Reified) s:owners:access s:confidence s:starts:durabilitys:location Privacy Situation P4 s:purposes:retention E3 E1 E2 s:creator s:method s:evidence E4 S4 s:position Explanation 2. Suggested Solutions Problem: the reified mainpart itself consist already of five elements, meta-reification? 12

13. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Extension of RDF-Triples to Quintuples Argument 1: need for different auxiliaries for each user model dimension –Peter is currently running –Peter likes running very much –Peter knows a lot about running Argument 2: need for different ranges for each user model dimension –Peter’s time pressure is high (within a scale of low-medium-high) –Peter’s time pressure is 1.5 (within a numeric scale between 0 and 2) –Peter’s time pressure is 88% (within 0% - 100%) From RDF-triples to quintuples: 2. Suggested Solutions [W3C - RDF] 13

14. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook From UserML-Predicates to the General User Model Ontology G UMO we don‘t want to redefine each predicate within each statement we don‘t want to define each predicate in an XML application each predicate concept contains default expiry information –physiologicalState.heartbeat: can change within seconds –mentalState.timePressure: can change within minutes –emotionalState.happiness: can change within minutes –characteristics.inventive: can change within months –personality.introvert: can change within years –demographics.birthplace: can’t normally change at all each predicate concept can change the default privacy settings –disabilities.colorblindness: should be accesible for adaptive presentations –disabilities.wheelchair: is interesting for pedestrian navigation systems –demographics.birthplace: accessible or hidden? –emotionalState.happiness: accessible or hidden? 2. Suggested Solutions 14

15. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Semantic Pointers from UserML to G UMO G UMO General User Model Ontology Subject = Peter Auxiliary = hasProperty Predicate = timePressure Range = low-medium-hig Object = high Start = 2003-04-16T19:28 End = ? Durability = few minutes Mainpart Situation UserML Statement semantic pointers Subject = Peter Auxiliary = hasProperty Predicate = timePressure Range = low-medium-hig Object = high Start = 2003-04-16T19:37 End = ? Durability = few minutes Mainpart Situation UserML Statement Subject = Peter Auxiliary = hasProperty Predicate = timePressure Range = low-medium-high Object = high Start = 2005-04-16T19:15 Durability = few minutes Mainpart Situation UserML Statement expiry+privacy defaults 2. Suggested Solutions time pressure semantic pointers = extended URIs, called UbisIdentifiers & UbisExpressions also pointers to other ontologies like SUMO/MILO and UbisOntology G UMO, defined in OWL and RDF cognitive load physiological state happiness 15

16. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Semantic Web Representation Predicate = rdf:Description Happiness 800616 minutes.520050 medium.640032 2. Suggested Solutions 16

17. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook User Model Auxiliaries and Basic User Dimensions (Classes+Intances) DEMO [Jameson,2001],[Kobsa,Koenemann,Pohl,2001][by Introspection] 2. Suggested Solutions 17

18. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Embedding G UMO: From Real World to UbisWorld Modeling Representational Models (Ontology, Schemata, Relations,...) Modeling Standards G UMO & UserML (Ontologies and Markup Languages) 3 Reduction & Abstraction UbisWorld for Ubiquitous User Modeling UbisWorld for Ubiquitous User Modeling 2 Real World with Ubiquitous User Modeling Real World with Ubiquitous User Modeling 1 Tools 4 Monitor / Control Simulation Representation / Use 2. Suggested Solutions 18

19. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook 19 G UMO as part of UbisOntology 3 4 7 21 3 45 Relations & Statements (binary or n-ary) 2 8 1 8 3 9 7 12 3 35 UbisWorld Concept ( Ontology + Instances + Relations) Physical Ontology Spatial Ontology Tempora l Ontology User/Group Activity Ontology Classes & Predicates Device/Object Location Individuals (id, label, category, parents, …) Time OWLDamlOIL RDFSQL Inference Ontology Inference Ontology EventSituationInference OWLRDFSQLXML Situation Ontology GUMO 2. Suggested Solutions T-Box A-Box * (distributed) based on Realism and Partiality, [Barwise] properties and relations are taken to be real objects not every detail is represented

20. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook 20 Spatial Concepts in UbisOntology 2. Suggested Solutions Building Info  Room Info  Region Info  Country Info  City Info  Part-of-Room Info  Floor-Level Info  Street Info  Object-Level Info  Handling of Objects Indoor Moving (In one Room) Indoor Moving (Between Rooms) Outdoor Moving Car & Bus Driving Airplane & Train Traveling 1000 km 100 km 10 km 1 km 20 m 5 m 1 m 10 cm Spatial Elements Location Spatial Constraints & Topology Spatial Granularity Levels Human Interaction (Mobility) Place Info  Spatial Granulation Level Entity Location Objekt Info  at / near Relat. Location Info  Location Spatial-Relat. Area Info  Location Extension Map Reference Info  Location Map Coordinates Connection Info  Location Connected Nesting Info  Location A contains B Constraint Info  Location Spatial-Relat. Orientation Location Info  facing

21. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook 3. Smart Situation Retrieval Information flow with UserML & UserQL Conflict Resolution Filter & Ranking User Model Service DB XML DB Distributed SituationalStatements Distributed Ontologies User Model Ontology Other Ontologies Situation Adder user/system-adapted UserML Report UserQL Request Add by UserML User 1 3 5 2 4.1 4.3 4.2 System User System RDF 3. Smart Situation Retrieval 21

22. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook 22 3. Smart Situation Retrieval Conflict Resolvers: mostRecent, mostNamed, mostConfident, mostSpecific, mostPersonal Adaptable Conflict Strategies: query.strategy arranges conflict resolvers in a list Smart Situation Retrieval with Conflict Resolution Conflict Resolution Conflict Sets S*A*P*R* UserML Report Document select UserQL Request Document FilteringResult Group Member Mapping Semantic Property Mapping Semantic Range Mapping Variation Mapping Remove Expired Remove Replaced function format XML DB Distributed Situational Statements RDFOWL 1 2 3 4 5 7 6 8 GUMO UbisWorld SUMO/MILO naming ranking match Smart Situation Retrieval filter

23. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook The U2M Approach with Tools and Aplicatons XML Distributed Statements Distributed Ontologies Other Ontologies DB RDF UbisWorl d Ontology + SUMO UserMode l Ontology GUMO Distributed Services Situation Server Conflict Resolution Situation Adder Ontology Reasoning Retrieval Filter Inference Engine Interface Manager OWL Applications Navigation PersonalNavi IndoorPosition (Real, m3i) Museum TorreAquila VölklingenIron (Peach) Speech GenderDetector AgeDetector (m3i) BioSensors AlarmManager BioRating (Ready, Rena) Shopping CyberShop SmartShop (Real, Specter) UserQL UserModel Editor XForms Viewer Privacy Editor Ontology Editor UserML Viewer Location Manager Tools UserML Add Query Report Context Editor 4. Implemented Systems [Kruppa] [Wasinger] [Stahl] [Baus] [Schneider] [Brandherm] [Müller] [Schwartz] […] 23

24. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Ubiquitous User Modeling meets the Layers of the Semantic Web Logic + Rules RDF + RDF Schema XML + XML Schema Unicode + URI Ontology Vocabulary Proof Trust UbisIdentifier & UbisExpression UserML/XML UserML/RDF G UMO + UbisWorld Conflict Resolution Explanation Privacy Applications and Extensions for Ubiquitous User Modeling Layered Architecture of the Semantic Web according to Tim Berners-Lee 5. Conclusion & Outlook 24

25. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Contribution, Impact and intended Future Work Contribution –integration of the three research areas: user modeling, ubiquious computing and semantic web –SituationalStatements, UserML, UserQL –G UMO & UbisWorld –Smart Situation Retrieval –Many tools for ubiquitous user modeling Impact –over 100 registered users in UbisWorld –new workshops about „ubiquitous user modeling“ –best poster award for UserML –G UMO is actively discussed in the user modeling community Future Work –evaluation of the developed user interfaces –semantic transfer between different ontologies (G UMO, SUMO/MILO) –idea to develop tools for individualized ontologies, based on G UMO –new project for the exchange of user modeling inference rules 5. Conclusion & Outlook 25

26. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Appendix A

27. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Integration of Situational Statements into the rest of the ontology N-ary Statements C3 3 D15 3 D42 7 5 3 D61 2 14 Binary Relations 145 3 ClassesProperties 2 P3 C6 C3 C5 C1 C2 C4 P4P2P1 Complex (C3 ∩ C6) U C5 ● Class Expressions DataTypes Integer, Float, String, URI, Enumerates,... Axioms ● Property Characteristics transitive, symmetric, inverse,... ● Class Characteristics disjoint,... T-Box A-Box M A ● Domain ● Range Individuals A ● Equiv? SameAs? D2D1D4D3D6D5 SubClass SuperClass SubProperty SuperProperty InstanceOf Instanciates RealizationOf Realizes domain range Property Inheritance Intance Inheritance A, M Abstract class multiple class physical, spatial temporal, events context, inference 3 4 3 51 221 Ontology Legend Category Legend [W3C - OWL]

28. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Centralized, distributed or isolated Mobile User Modeling? UM 1 UM 2 UM Centralized Mobile User Modeling Isolated Mobile User Modeling Distributed Mobile User Modeling Situation A Situation BSituation C Situation DSituation E Situation F UM - Limited Device Resources - Inference Approximation - Synchronization Problem - Communication - Network Dependency - Server Dependency

29. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Our Model of Situated Interaction

30. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Group Attribute Models

31. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Theories for basic user model dimensions Personality Traits (see Gumo) –OCEAN, Five Factor Model, [McCrae & Costa,1992] Openness, Conscienciousness, Extraversion, Agreeableness, Neuroticism can be dervied from factor analysis reports –PEN, Three Factor Model, [Eysenck & Eysenck,1991] Psychoticism, Extraversion, Neuroticism is biologically based (on heart beat, blood pressure, sweating,…) –MeyersBriggs Type inventory –Interpersonal Theory [Leary,1957] deals with peoples interaction patterns circumplex (dominant | submissive, hostile - friendly), complementary, vector length Mood + Emotion (see dissertation [Patrick Gebhard,2006]) –OCC Model, Cognitive Appraisal Theory, [Ortony, Clore, Collins] popular in affective computing, explaining the generation of emotions –PAD Model, Pleasure-Arousal-Dominance Temperament Framework [Mehrabian,1996], three dimensions for moods Interpersonal Relations –Balance Theory [Heider,1958], [Markus Schmitt,2005] combines cognitive elements to explain interpersonal relations balanced and unbalanced cognitive (triangle) structures: P-O-X Schmitt extends this theory and develops a dynamic attitude model that simulates changes in the interpersonal relations as a result of communication processes [Klesen,02] [Oberander,04] [Picard,97] [Eysenck,91],[McCrae,Costa,92] PO X

32. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Implicit Manual Input as Source for Upward Inference in User Modeling observesinstantiates models adapts to (high force) (wrong place) (too far) (high) uses influences Symptomatic behaviors Resource limitations Time pressure Cognitive load Lack of knowledge Type Click Scroll has User [Lindmark,Heckmann,2000] … to interpret symptoms of cognitive load and time pressure

33. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Design of the Exchange Language? User Property User Model User PreferencesUser Demographics General Interests Psychological State Behavioral Regularities User Plans AddressAgeSex User Knowledge and Belief Working Memory Load Timepressure Hobbies Privacy Interface LayoutNewsTypingModality Physiological States Pupils DilationBloodPressure User Role (Maximal) XML with a built-in User Model Taxonomy high (Medium) XML with commonly accepted terms and pointers Pointer-to-Ontology high (Minimal) XML with pointers to User Model Ontology Pointer-to-Ontology Pointer-to-Range

34. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Privacy in User Adaptive Systems There are four main arguments, (according to Kobsa) that influence the users’ decisions about allowing personalization with their personal data. 1. What will be done with my personal data? –This question focuses on the purpose/intention 2. Who is going to use my personal data? –This question focuses on the access and the recipient 3. Which kind of personal data is used? –Thus a differentiation between the type of data is implied. 4. In which mood or situation is the user currently? –This last point suggests ”user-adaptive useradaptivity” The most important privacy treatment is that the user should be able to ”control” the systems’ private information handling. One point will be the inspection of the stored data, another point will be the possibility to change it. Additionally, the exchange of privacy settings is important.

35. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Basic Privacy Treatment in U2M: Trust-based + Scrutiny Interfaces Statement key owner access purpose retention Query key requestor intention ”Purpose: commercial” means that this information can be used for commercial purposes. ”Purpose: research” means that this information should not be used for commercial purposes. ”Purpose: minimal” means that this information can only be used by the owner himself. ”Retention: long” must be deleted within a year. ”Retention: middle” must be deleted within a month. ”Retention: short” must be deleted within days. ”Access: public” means that everybody can be the recipient of this information. ”Access: friends” means that only friends can be the recipient of this information. ”Access: private” means that only the owner can be the recipient of this information.

36. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Integration Schema for user modeling & context-awareness Context Model (aspects of the context) Context Model (aspects of the context) New Information about the user / context New Information about the user / context Decision Hypothesis about the user / context Decision Hypothesis about the user / context Output Downward Inference (Model Application) Downward Inference (Model Application) Upward Inference (Model Acquisition) Upward Inference (Model Acquisition) Any Information about the user / context Any Information about the user / context Sensor Information about the user / context Sensor Information about the user / context Input Model Personal Journal (with contextual parts) Personal Journal (with contextual parts) User Model (aspects of the user) User Model (aspects of the user)

37. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook The steps to User Model Integration HTTP + XML + URI + SOAP + RDF + OWL UbisIdentifier (Ubid) + UbisExpression (Ubex) Situational Statements (UserML) + Situational Queries (UserQL) User Model Ontology (GUMO) + Ubis World (UbisOntology) Ubiquitous User Model Service (U2M) + User Model Adder Smart Situation Retrieval + Conflict Resolution (S * A * P * R * ) User Model Integration + Decentralized User Modeling 7 6 5 4 3 2 1

38. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook C.Hallway center of Hallway WW Info  C.Room 124 center of Room 124 Info  Room 124 Info  Hallway Info  Topology: Nesting and Connections Campus Info  Mensa Info  Building 36 Info  WW Floor Info  Dining Hall Info  2nd Floor Info  Info Key 124 Distance3 m Admission Stairsno connected Connection Info  C.Room 124 C.Hallway Spatial Granularity Levels: Part-of-Room / Room Info Key 124 Distance 50 m Admission Stairs yes connected Connection Info  Dining Hall Room 124 Spatial Granularity Levels: Room / Floor / Building

39. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Hybrid Location Modeling Map-Referencing Map Reference mapping Info  Building 36 Campus Map Coordinates Spatial Granularity Level: Street - Building Building 36 Info  Campus Info  Building 45 Info  Map of University Campus Building 28 Info  Building 42 Info  Map Reference mapping Info  Room 124 Floor Map Coordinates Room 124 Info  Hallway Info  Kitchen Info  Map of WW-Floor WW Floor Info  Spatial Granularity Level: Floor - Room

40. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook UserModel Integration in U2M SQL XML RDF IN1 SQL XML RDF M1 S R=M1,S=1  S R=M1,S=… S R=IN1,S=1  S R=IN1,S=… Upward Inference 1 SQL XML RDF M1 S R=M1,S=1  S R=M1,S=… SQL XML RDF M2 S R=M2,S=1  S R=M2,S=… SQL XML RDF M1◦M2 S R=M1◦M2,S=1  S R=M1◦M2,S=… Merge Inference SQL XML RDF IN2 S R=IN1,S=1  S R=IN1,S=… SQL XML RDF M2 S R=M1,S=1  S R=M1,S=… Upward Inference 2 SQL XML RDF IN1 SQL XML RDF M1 ◦M2 S R=M1◦M2,S=1  S R=M1◦M2,S=… S R=IN1,S=1  S R=IN1,S=… Upward Inference 1 ◦ 2 SQL XML RDF IN2 S R=IN2,S=1  S R=IN2,S=…

41. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook User-Adaptivity, Resource-Adaptivity and Context Awareness Conceptual Partition of the World User Adaptivity Rest of the World System/DeviceUser User-adaptive Interaction User Adaptive i.e. colour blindness Rest of the World System/DeviceUser Resource-adaptive Interaction Resource Adaptivity Resource Adaptive i.e. low battery Rest of the World Context System/ Mobile Device User Context-aware Interaction Context Awareness Context Aware i.e. noisy environment

42. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Situated Interaction and Ubiquitous Computing Conceptual Partition of the World Situation Awareness Context-Aware Intelligent EnvironmentUser Ubiquitous Computing User Adaptive Context-Aware Resource Adaptive Situation-Aware Human-Environment Interaction Situation-Aware Human-Computer Interaction i.e. colour blindnessi.e. low batteryi.e. noisy environment User Adaptive Resource Adaptive Situated Interaction Adaptation Partitioning of the World Rest of the World EnvironmentSystem / DeviceUser Situated Interaction Adaptation Partitioning of the World Neighbour & History-Aware

43. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook Conflict Resolution Strategies mostRecent(n) Especially where sensors send new statements on a frequent basis, values tend to change quicker as they expire. This leads to conflicting non-expired statements. The mostRecent(n) resolver returns the n newest non-expired statements, where n is a natural number between 1 and the number of remaining statements. mostNamed(n) If there are many statements that claim A and only a few claim B or something else, than n of the ”most named” statements are returned. Of course it is not sure that the majority necessarily tells the truth but it could be a reasonable rule of thumb for some cases. mostConfident(n) If the confidence values of several conflicting statements can be compared with each other, it seems to be an obvious decision to return the n statements with the highest confidence value. mostSpecific(n) If the range or the object of a statement is more specific than in others, the n ”most specific” statements are returned by this resolver. mostPersonal(n) If the creator of the statement is the same as the statement’s subject (a self-reflecting statement), this statement is preferred by the mostPersonal(n) resolver. Furthermore, if an is-friend-of relation is defined, statements by friends could be preferred to statements by others. 4. Smart Situation Retrieval

44. 1. Motivation & Background 2. Suggested Solutions 4. Implemented Systems 3. Smart Situation Retrieval 5. Conclusion & Outlook GUMO Elements by: Literature Study, Jameson`s Tutorial, Introspection, …