1 Formal Specifications of Image Schemata for Interoperability in Geographic Information Systems Andrew Frank Department of Geoinformation Technical University Vienna

2 Outline My background Spatial Cognition is Crucial Need to Formalize Semantics of Spatial Prepositions Principles of our Investigation Relations in Geographic Space Relations in Table-Top Space Current work Future: Cognitive Engineering

3 My background Diploma in surveying engineering ETH Zurich Doctoral work on databases for spatial information ETH Zurich Research in GIS at University of Maine One of leaders of the National Center for Geographic Information and Analysis (NCGIA - UCSB, SUNY Buffalo, UMaine) Professor for Geoinformation, TU Vienna Initiation of series of conferences on Spatial Information Theory (COSIT)

4 My past research Database management Formal data modeling methods Formalization and programming of geometric operations in a realistic environment (limited precision, rounding problems, errors) Bridging between formal research in geometry and cognitive and geographic investigation in human understanding of space (with David Mark)

5 Spatial Cognition is Crucial O’Keefe/Nadel Hippocampus serves to construct a neural map – (similar to the mental map/collage - B. Tversky) Humans use the left hippocampus for verbal problems Methods originally developed for spatial problem solving are transferred to the general (abstract) verbal domain –navigation of semantic nets as a fitting metaphor

6 Spatial Cognition is Crucial (Lakoff) Spatial metaphor abound in language (especially English) –we are at crossroads, I’m feeling down Bodily experience provides grounding for the semantics of simple (primary experience) elements of language (Johnson) Metaphorical transfer links meaning of abstract terms to the bodily experience (Lakoff)

7 Practical Need to Formalize Semantics of Spatial Prepositions Interoperability of GIS => Co-operation of different GIS Requires: Standardization of Terms: Semantics Problem Different communities use terms very differently: –width of a road –parcel –wood

8 Practical issues Building of integrated European Land Use Databases (necessary for discussion of potential political action) Integrated Database for Town Administration: “one-stop” administrative procedures All require semantic integration from multiple domains Standardization efforts of industry: –need to identify the building blocks that are universal

9 Principles of Investigating Spatial Relations Language allows to study (aspects of) spatial cognition: Consider verbally expressed situations and what is implied (can be deduced) from the description (not just simple sentences). Restrict to non-metaphoric usage (Montague?) Very specific environment (only a single ‘space’, e.g., table- top space; to have influences from only one kind of bodily experiences) Assume polysemy liberally (identification later)

10 Spatial Prepositions investigated Geographic Space and Table-Top Space Exclusion of Partial Spatial Relations Single Level of Detail Specific Environment and Single Language –surprising amount of diversity within a single language (regional diversity) German: auf dem Tisch, Vienna: am Tisch

11 Previous efforts: Formalizing Spatial Meaning Topological Relations based on formal considerations and some evidence of cognitive soundness (Egenhofer, Mark) Metric Relations (Cardinal Directions; discretized directions: Near, Far, etc.) What is universal? How much is left? Language encodes many other spatial relations (typically 50 to 100)

12 Method of Formalization Formalize the meaning of several related spatial preposition (above/under) Static spatial prepositions can be modeled with (first order) predicate calculus (logic) If one includes movement and changing situations (including Talmy’s fictive motion) algebraic approaches lead to simpler formalizations. Need for a tool to maintain facts about the environment (‘database’)

13 Executable Formalizations Formalizations are difficult to read and understand - and often contain errors. We currently use a formalization system (based on denotational semantics, category theory) packaged in a functional programming language. Our formalization is formally checked (for completeness, type conformance etc.) and can be ‘tested’ (run) – do they produce the predicted output?

14 Are Related Spatial Prepositions Image Schemata? Image schemata are: Recurring, Imaginative Patterns based on People‘s Experiences Sufficient Internal Structure to Constrain People‘s Understanding and Reasoning More Abstract than Mental Pictures, Less Abstract than Logical Structures Source for metaphorical usage Accepted as a pragmatic concept (notion) for our research

15 Image Schema PATH

16 How to Formalize Spatial Image Schemata Predicate Calculus –A above B => B under A Relation Calculus a (R;S) c = aRb and bSc –A north of B and B north of C => A north of C Functions f.g(x) = f(g(x)) –move out (Coin, (move in (Coin, Box) ) = isOutOf (Coin, Box) Model Based

17 Example Geographic-Space-Image- Schemata LOCATION PATH REGION BOUNDARY

18 Select Base Relations Location in Region Region inside Region Location directly connected to Location Region borders Region –formal (but not cognitive) justification for the selection of base relations

19 Location and Relation between Places Direct and Indirect Path path (a, b) path (b, a) ind-path (a, b) [path (a, a1) & path (a1, a2) & path (a2,...) &... & path (..., bn) & path (bn, b)]

20 Relations with Region Location within Region in* (loc1, reg2) in (loc1, reg1) & in (reg1, reg2)

21 Relations with Boundaries Boundary Locations boundaryBetween (a, b) path (a, b) & notInSameRegion (a, b)

22 Persons Move scene2 = move (place1, scene1) move (p, a, b): in (p, a) & path (a, b) unestablish (in (p, a)), establish (in (p, b))

23 Conclusions ‘Geographic Space’ Extension of Relation Calculus to Function Calculus 5 Base Relations => 15 Meaningful Relations Formal Spatial Relations => Interoperability, Optimizing Spatial Queries Powerful Domain as Source for Metaphors

24 Examples from Table-Top Space (1) „In“ Blocks Target of Movement x ‘in’ y (in scene) => blocked (move z into x (in scene))

25 Examples from Table-Top Space (2) Converse of „auf“ Blocks Object of Movement x ‘auf’ y (in scene) => blocked (move y in scene)

26 Conclusions ‘Table-Top space’ Some relations are similar to geographic space; polysemy could be dropped and core of meaning identified –(problems with formalization) Limit to specific environment is fruitful; next steps: –spatial relations in image space –spatial relations in city space Expectation: full formalization of all German spatial prepositions

27 Our related research Models of environments and observers Simulate language expressions describing the environment for various observers –problem perspective taking: “from your point of view, the coin lies just behind the box” Formalized concepts often described as deictic, intrinsic, absolute etc. Open issues: build models for effects like ‘imagined translation vs. imagined rotation’ (Klatzky)

28 Current Work: Spatial Affordances (student: Martin Raubal) Formal description what we understand by ‘affordance’ Models of actors (persons) with tasks and objects Formalization as models of complex environments including actors having their own observation function and maintaining their own (temporal) knowledge base Open issue: effects of hierarchies in space and in the tasks (Timpf) (Formal complication: databases in databases…)

29 Practical Application Predict navigation performance in complex built environments, in order to improve the built environment. Meaning: –improve signage in airports so you find your gate and make it unlikely that you get lost in the transfer!

30 Current Work: User Interface (with Werner Kuhn) Metaphor is keyword in graphical user interface research, but not well defined Formalize metaphor to construct metrics to assess the ‘cognitive load’ for learning and usage of an user interface Apply to design better user interfaces!

31 Future: Cognitive Engineering Cognitive science can be successfully applied in a predictive mode: engineering Some differences in the methodology and approach between engineering and science: –concentrate on primary influence before investigating secondary effects –simplify problems and subdivide issues (Occam?) –formalize to allow use in a predictive mode

32 Open Questions Best Method for Formalization Are there Language Independent Primitives? Composition and Interaction of Image Schemata Integration of Image Schemata Across Domains Formalize Metaphorical Transfer