Ontologies for Reasoning, Action and Interaction in Space John Bateman University of Bremen SOCop Meeting: 12th November 2009

Overview of talk Context of the work Representations of Space within our Collaborative Research Center: “Spatial Cognition”... ... and my work within that Representations of Space Results and Conclusions Proposals for Ontology Best Practice Next targets and challenges

Spatial Cognition: SFB/TR8 Areas: spatial reasoning, representation, action and communication Organisation: Collaborative Research Center Bremen / Freiburg funded by: Deutsche Forschungsgemeinschaft (DFG: German Research Council ) 3 Phases 1st phase: 2003-2006: 12 projects 2nd phase: 2007-2010: 18 projects 3rd phase: being proposed

Spatial Assistance Mobility support Spatially-embedded tasks Descriptions of spatial situations verbally visually Exploration Navigation Dialogic interaction

Bremen Autonomous Wheelchair: Rolland Qualitative Information “In front to the right is the seminar room” Quantitative information : Rolland Symbolic information [door_1 recognized]

Test-Bed: BAALL Bremen Ambient Assisted Living Lab Environment heating, lighting safety soft / hard / middleware interaction Appliances refrigerator cooker cupboards, drawers washing machine, microwave TV, PC, … mobile phone doors Autonomous assistance devices wheelchairs walkers Health monitoring Architecture

Sensor data: ‘free-space’ maps

Voronoi calculation on a scanned floor plan “where are you?” Voronoi map From SFB/TR8 project: A1-[RoboMap]

Unnatural / unhelpful descriptions 25.4 m NW of you GPS: “34° 15´ N / 3° 27´ E” “3.45m away from edge 98 (with 80% certainty)”

Natural route descriptions Leave the room and turn right into the corridor. Go to the window and then turn left. Follow the corridor and I’m in the last room on the left. Many problems of semantic interpretation involved here...

Context-specific way-finding assistance schematization Kai-Florian Richter, Thomas Barkowksy et al. "At the next junction go straight on, and then, turn right before a map. Keep following the river until a Telekom sign, and then, turn left after the Telekom sign. Go towards the Universum until a bus stop, and then, turn right after the bus stop."

Sources of relevant knowledge Location-based services Geographic Information Systems Commonsense objects and activities Spatial awareness and understanding Natural language capabilities Robot perception

Basic problem Getting these diverse areas of expertise to talk to each other is a serious issue different communities different interests different representations The kinds of knowledge maintained by such systems are very different

Consequences Component specifications are developed again and again (and again and again) Each community of practice defines them slightly differently (or not, it is difficult to tell) Each standardisation group has little time to look at parallel activities and must reflect the demands of its own community before considering others Lack of foundation leads to a proliferation of ‘standardisation’ efforts

15 Research Foci: John Bateman Computational Linguistics Multilingual natural language generation Interfaces between language technology and world/domain knowledge Development of linguistically-motivated ontologies Formal Ontology General design principles for ontology Relations between differently motivated ontologies Spatial representation and language Dialogic natural language interaction Spatial language 1988 1992 2001 15

Solution we are pursuing High degree of interoperability between diverse knowledge-rich systems is to be achieved by ontological engineering, taking in: knowledge of the human world (commonsense) knowledge of the robot world (programmed, emergent) geo-knowledge (GML, other standards) spatial knowledge (spatial calculi, spatial ontologies) knowledge of language (linguistics)

HIGHLY STRUCTURED AND MOTIVATED SEMANTICS Goals HIGHLY STRUCTURED AND MOTIVATED SEMANTICS Achieving interaction between system modules using ONTOLOGIES ONTOLOGY 1 ONTOLOGY 2 inter-ontology mediation ONTOLOGIES DOMAINS

Goals Achieving interaction between system modules using ONTOLOGIES SPATIAL REPRESENTATIONS LINGUISTIC REPRESENTATIONS inter-ontology mediation ONTOLOGIES DOMAINS

Goals Achieving interaction between system modules using ONTOLOGIES SPATIAL REPRESENTATIONS GENERAL ONTOLOGY inter-ontology mediation ONTOLOGIES DOMAINS

Goals Achieving interaction between system modules using ONTOLOGIES ROUTE GRAPH R3: GVG ROUTE GRAPH A1: PATH inter-ontology mediation ONTOLOGIES DOMAINS

Ontologies... ... but where to start?

Fundamental issue The ontologies present are diverse: different methodologies different motivations different domains of application different worlds different purposes different communities

Representations of Space Ontology and Space Qualitative Spatial Representation and Reasoning Language GIS

Representations of Space ontology Foundational Ontologies Qualitative Spatial Reasoning + Representation Linguistics physical mathematical Geometry R3

Ontology + QSR: Varying primitives Bateman/Farrar (2006) Spatial Ontology Baseline 25 25

Spatial Representations Ontology Qualitative Spatial Reasoning and Representation Language 26 26

Ontologies: SUMO 27 27

Ontologies: SUMO 28 Shape: internal attribute (inheres in some entity) Position: relational attribute 28 28

Ontologies: SUMO Spatial Relations 29 29

Ontologies: Cyc 30 30

Ontologies: Cyc 31 31

Ontologies: Cyc 32 32

Ontologies: Cyc: Paths 33 33

Navigation: route graphs MZH 3rd Floor kitchen MZH 3100 MZH 3110 Route Graph 34 34

Ontologies: DOLCE basic categories 35 35

DOLCE basic categories Rooms, offices, buildings, tables, chairs, ... 36 36

DOLCE basic categories Events, happenings, movements, changes 37 37

DOLCE basic categories Agents, states of belief, plans, goals 38 38

One-sided generic constant dependence DOLCE inter-entity relationships Feature (F) Non-Agentive Physical Object (NAPO) Amount of Matter (M) Physical Endurant (PED) Physical Quality (PQ) OGD GK MSDS Mutual specific spatial dependence One-sided generic constant dependence Generic constant constitution Quality (Q) heavy pebbly wood rocks, sand, water table coast corner of a table curve of a bay 39 39

Spatial Representations Ontology Qualitative Spatial Reasoning and Representation Language 40 40

QSR: Region-Connection Calculus with 8 base relations Randell, Cohn, Cui 1992: RCC8 41 41

QSR: Region-Connection Spaces: RCC-5, RCC-8, 9+, etc. Randell, Cohn, Cui 1992 etc. / Egenhofer 42 42

QSR: Reasoning by Composition Composition table for RCC-8 43 43

QSR: Double cross calculus: Freksa / Zimmermann (1996) Qualitative description of position relative to a directed line segment 44 44

QSR: Reasoning by Composition ab:c ○ bc:d →ab:d Composition for reasoning 45 45

QSR: Reasoning by Composition Homing, Shortcut, Inverse, Homing-Inverse,Shortcut-inverse Composition with additional relations 46 46

QSR: Star Calculus Renz/Mitra 2004 47 47

QSR: Dipole Moratz et al. 2000 48 48

QSR: OPRA Moratz et al. 2005 49 49

QSR: QTC 50 Van de Weghe et al. Qualitative Trajectory Calculus single object moving two objects moving 50 50

Spatial Representations Ontology Qualitative Spatial Reasoning and Representation Language 51 51

Linguistic usage evidence… what does ‘on’ mean? Herskovits (1986) 52 52

And more usage evidence… what does ‘in’ mean? Herskovits (1986) 53 53

And more usage evidence… what does ‘in’ mean? “The cat is in the table” Herskovits (1986:125) 54 54

And more usage evidence… what does ‘in’ mean? The potato is in the bowl Herskovits (1986) 55 55

Relating calculi and language Egenhofer and colleagues 56 56

Spatial Language extremely flexible sensitive to function and purpose 57 Coventry, Garrod and others 57

Many types of spatial information Ontology Qualitative calculi: RCC-n, Dipoles, Doublecross, etc. Way-finding abstractions: choremes Free-space representations (Voronoi) Natural language descriptions Metric maps With different reasoning methods, different coverage, different strengths and weaknesses 58 58

Representations of Space ontology Foundational Ontologies Qualitative Spatial Reasoning + Representation Linguistics physical mathematical Geometry R3 BFO DOLCE GFO RCC DC OPRA 9+ GUM-Space ? ? ? ? ‘alignment’ 59 59

Methodological conclusion and starting point There is no sense in which a simple ‘merging’ of all of the above is a sensible strategy to follow 60 60

Distinct facets or ‘theories’ rather than inheritance link in transit system (ferry) geographical region lake source of pure water recreational area obstacle 61 61

Many perspectives on ‘reality’: many ontologies space-1 event time space-2 event Ontologically diverse 62 62

Ontological diversity inter-ontology mappings landmarks event types time choremes CASL CASL CASL CASL route graphs Way description CASL 63 63

Mapping between modules “Hyperontology” points of interest road conditions time directions Mapping between modules health status problem area 64

Essential properties we are currently developing Perspectivalism Objects Activities Artifacts: spatial artifacts Language Granular partitions Plug-and-play spatial theories 65 65

Essential ingredients we are drawing on Existing ontologies Existing formal tools Extensions for the specific problem of combining information flexibly Combining distinct reasoning possibilities 66 66

Essential ingredients we are drawing on Existing ontologies DOLCE (for cross-category binding and axiomatization) BFO (for sites, niches and places and for SNAP/SPAN) GUM (generalized upper model for linguistic semantics) 67 67

DOLCE basic categories 68 68

Defining Qualities 69 I1-[OntoSpace] Quality Space Gärdenfors: Geometric Fauconnier: Logical 69 69

Defining Qualities 70 I1-[OntoSpace] Quality Space Gärdenfors: Geometric Fauconnier: Logical Goguen: algebraic theory Theory 70 70

Color Space (1): Color wheel HUE ORANGE YELLOW CHROMATICNESS GREY RED GREEN VIOLET 71 From: Gärdenfors (2000, p10) 71

Color Space (2) 72 WHITE GREEN YELLOW CHROMATICNESS BLUE RED From: Gärdenfors (2000, p11) BLACK 72 72

Defining Qualities 73 Quality Space 73 WHITE GREEN YELLOW BLUE RED BLACK WHITE 73 73

Dolce treatment of qualities Qualia: the position of an individual quality within a quality space 74 74

DOLCE: spatial information For DOLCE, space is also a quality... 75 75

DOLCE: relevant for space Physical Objects Physical Endurants (PED) Spatial Location Space Region 76 76

DOLCE: relevant for space Physical Objects Physical Endurants (PED) Spatial Location Space Region 77 77

Dolce treatment of qualities Qualia: the position of an individual quality within a quality space 78 78

Dolce treatment of qualities Space Region Qualia: the position of an individual quality within a quality space 79 79

Valuable for `swappable’ treatments of space ql PQ PED qt Physical Endurant Physical Quality Quality Space: quale 80 80

`Swappable’ treatments of space Should be possible to select formalization for the reasoning task at hand Formalized modules ql PQ PED qt RCC-5,7,8,10,15,23 Dipoles: D14 , DRA14 DRAfp Cardinal directions Double Cross 81 81

Essential ingredients we are drawing on Formal and computational tools CASL Common Algebraic Specification Language (for specification, structuring and relating) HETS Heterogeneous Tool Set (for connecting to a range of reasoners) 82 82

Formalization choice: CASL Common Algebraic Specification Language Standardised first-order specification language designed by CoFI “Common Framework Initiative for algebraic specification and development” since 1995 de facto standard approved by IFIP WG 1.3 “Foundations of Systems Specifications” (1998), extensive documentation (LNCS 2900, 2960) extensive User Manual and Reference Manual now available from Springer (LNCS 2900, LNCS 2960) supports structured specifications including imports, hiding, renaming, union, extensions, etc. 83

The Tool HeTS Institution Theory 84

CASL language constructs Structuring mechanisms: potentially applicable to any specification language as an additional layer of ‘meta’-organisation for semantic modularity OWL, UML, CommonLogic, etc. 85

Formalization choice: CASL Common Algebraic Specification Language Extensions: we have now added OWL-DL to the family of logics supported we are exploring combining the structuring principles of CASL and description logics we are progressively formalizing the entire family of qualitative spatial calculi 86 86

Lüttich & Mossakowski (FOIS 2004) Axiomatized Ontology in CASL DOLCE Primitives GenParthood 87 87

Lüttich & Mossakowski (FOIS 2004) GenParthood DOLCE GenMereology 88 88

Lüttich & Mossakowski (FOIS 2004) Primitives GenParthood GenMereology spec MEREOLOGY = PRIMITIVES then %%Ad7, Ad8, Ad9 and Ad10 are generated by %% instantiation of GenMereology GENMEREOLOGY [sort T] GENMEREOLOGY [sort S] GENMEREOLOGY [sort PD] end Mereology 89 89

The DOLCE ontology in CASL spec PreDolce = Mereology_and_TemporalPart and Temporary_Mereology and Participation and Constitution and Dependence and Direct_Quality and Temporary_Quale and Immediate_Quale end spec Dolce = PreDolce and Taxonomy end work continuing... 90 90

Lüttich & Mossakowski (FOIS 2004) Development Graph showing dependencies between specifications and proof obligations Links: theory morphisms imports of theories relative interpretations of theories open proved 91 91

Reasoning 92 First-Order Reasoning with CASL/HETS reasoners Description logic reasoning with DL reasoners Spatial Reasoning with specialized spatial reasoners: SparQ, GQR 92 92

Reasoning: SparQ Frank Dylla, Lutz Frommberger, Jan Oliver Wallgrün, and Diedrich Wolter 93 93

Lessons drawn Idea: Providing channels to ontologies provides access to detailed contextual ‘world-knowledge’ that does not then have to be worked out again… Ontology Application 94 94

Sharing knowledge and achieving interoperability Many projects, many products, many information providers now constructing ontologies BUT: proliferation of unrelated designs, impoverished or application-specific semantics, ‘roll your own’ ignoring previous attempts lack of interoperability ... which was precisely what ontologies were meant to provide! 95

modelling language dependence Metamodels: commonly restricted to ensuring translatability across formal languages not content modelling language dependence Horiuchi 96

LOA, Dolce group: EU WonderWeb Project 97 97

Problems... 98 Looseness of definition Sparseness of definition does not give much to ‘get hold of’ for relating distinct accounts/levels of abstraction 98 98

Conclusions: Ontology Best Practice ‘Light’ ontologies: semantic web ... ‘Heavy’ ontologies: Rich axiomatization Formal principles Well-defined design criteria 99 99

Lessons drawn 100 Ontological best design principles axiomatization modularity heterogeneity perspectivalism 100 100

Ontology construction Axioms are grouped into logically appropriate theories Theories may be extended via parameterization to achieve semantic re-use Theories may be created and related by views: theory morphisms Only with this availability of working with meaningful interrelationships can the complexity of distinct axiomatized ontologies really be harnessed. 101 101

Where next? Geospatial Information set to become the next major area of ontological development? However, just converting existing schema to OWL is probably not going to be adequate 102 102

Questions: modularity? OpenGIS® City Geography Markup Language (CityGML) Encoding Standard (2008) 103 103

Questions: modularity? 104 Geometry to BDM to IFC 104

Selected Application Scenarios Our Next Steps Selected Application Scenarios assisted ambient living (AAL) ADL, spatial activities, ... geographic information science (GIS) OpenGIS, OGC, CityML, OpenStreetMap, ... assisted architectural design (AAD) IFC, BIM, ... In each application area, we want to interact directly with the appropriate national and international standards 105 105

Geographic Information Systems <way id="34243185" visible="true" …> <nd ref="59714051"/> … <tag k="landuse" v="forest"/> </way> Our Next Steps Geographic Information Systems Ontologies framework for connecting distinct geographic layers modular breakdown of relevant knowledge improving re-use relation to non-geographic modeling relation to qualitative representations relation to existing standards support for verbalisation and visualisation <way id="27562683" visible="true" …> <nd ref="302109295"/> … <nd ref="302109288"/> <tag k="name" v="Feldackerweg"/> <tag k="surface" v="paved"/> <tag k="maxspeed" v="30"/> <tag k="highway" v="residential"/> </way> <way id="34243185" visible="true" …> <nd ref="59714051"/> … <tag k="landuse" v="forest"/> </way> OpenStreetMap 106 106

and we can only do that in cooperation with those with the detailed expert knowledge! 107 107

Acknowledgements 108 The entire SFB/TR8 team! http://www.sfbtr8.uni-bremen.de 108 108

Advertisement International Research Training Group: semantic integration of geospatial information http://irtg-sigi.uni-muenster.de Application deadline soon! PhD training program 109 109