1. Integrating Multi-Media with Geographical Information in the BORG Architecture R. George Department of Computer Science Clark Atlanta University Atlanta, GA

2. Outline The BORG Architecture –Where We Fit In Spatio-Temporal Query Models Applications –The Weather Information and Tactical Support (WITS) System –Integrating Multimedia with GIS Conclusion

3. Battlefield Organic Robotic Grid Ubiquitous Knowledge Environment for the Battlefield Built upon an ad-hoc computing environment configured as a single computing resource

4. BORG Experimental Architecture & Platform

5. Data Related Issues Heterogeneous Data Sources Multiple Data Types Flexible Querying with Levels of Confidence Need for Semantic Queries Performance

6. Research Objectives Spatial, Temporal Dimensions Features of Data –Characterization of Spatio-Temporal Query Types Applications: –Support for Operational Planning Long term weather patterns Detect weather anomalies not predicted by forecast models Integration with External Knowledge Bases –Situational Awareness in an Urban Environment Integration of Heterogeneous Databases including Multi-Media Approach: Fuzzy Logic provides a expressive query mechanism for the spatial and temporal domains

7. Spatial Properties Minimum Bounding Rectangle basis for Query –The location of an object Ai is defined by the rectangular area (a region) [(Xi, Yi), (Xj, Yj)] where Xi  Xj, Yi  Yj. –The spatial property of an object (of interest) A is a tuple (R, I), where, R is a rectangular area (a region) which covers a minimal area in which object A appears during the time interval I = [ti, tf]. R is an approximation of a MBR. –A static or moving in the rectangle R during the interval I. When A is spatially static, R is the minimum bounding rectangle of A.

8. Relationships between Objects Defined using the Spatial Relationship between MBRs. Extension of Allens Temporal Interval Algebra [Li et al]

9. Relationships between Objects RelationDefinition A BOTTOM B Ax {b, bi, m, mi, o, oi, d, di, s, si, f, fi, e} Bx  Ay {b, m} By A TOP B Ax {b, bi, m, mi, o, oi, d, di, s, si, f, fi, e} Bx  Ay {bi, mi} By A LEFT B Ax {b, m} Bx  Ay { b, bi, m, mi, o, oi, d, di, s, si, f, fi, e} By A RIGHT B Ax {bi, mi} Bx  Ay { b, bi, m, mi, o, oi, d, di, s, si, f, fi, e} By A TOP-LEFT B (Ax {b,m} Bx  Ay {bi, mi, oi} By )  (Ax {o} Bx  Ay {bi,mi} By) A TOP-RIGHT B (Ax {bi, mi} Bx  Ay {bi, mi, oi} By )  (Ax {oi} Bx  Ay { bi,mi }By) A BOTTOM-LEFT B (Ax {b,m} Bx  Ay {b,m,o} By )  (Ax {o} Bx  Ay {b,m} By) A BOTTOM-RIGHT B (Ax {b,m} Bx  Ay {b,m,o} By )  (Ax {oi} Bx  Ay {b,m} By) A OVERLAPS B Ax {d, di, s, si, f, fi, o, oi, e} Bx  Ay { d, di, s, si, f, fi, o, oi, e } By A EQUAL B Ax {e} Bx  Ay {e} By A INSIDE B Ax {d} Bx  Ay {d} By A CONTAIN B Ax {di} Bx  Ay {di} By A TOUCH B (Ax {m, mi} Bx  Ay {d, di, s, si, f, fi, o, oi, m, mi, e} By)  (Ax {d, di, s, si, f, fi, o, oi, m, mi, e} Bx  Ay {m, mi } By) A DISJOINT B Ax {b, bi} Bx  Ay {b, bi} By

10. Relationship between Objects The fuzzy spatio-temporal relationship during time interval, I is Ai ( , , I) Aj –  is a relation;  is the value of membership and Ai (   )Aj is true during the interval I. –Ex:  takes values of WEST, NORTH, NORTH-WEST

11. Query Model Query is performed on regions within the underlying data structure that form a Minimum Bounding Rectangle. Query Model supports queries in the spatial and temporal dimensions –Spatial Search:  S, retrieves all regions, Rj, whose area is equal to that of the user selected MBR, Ri, and field values are similar to those of Rj at Time, Tk  S (MBR) : {Rj | Equal(Ri, R) Λ Disjoint(Ri, Rj) Λ ( ,  ≥ ε)} Tk –Temporal search :  , retrieves region, R, within a time interval, (T0, Tn), to retrieve all instances in which the attribute values are similar. The spatial domain is constant in this operation.   (MBR) = {Tj, R | Equal(Ri, R) Λ ( ,  ≥ ε)} T0-Tn –Spatio-Temporal Search:  ST, identifies similar regions in space and time. The MBRs adjacent to the original are examined to track the weather in the spatial domain. The coordinates of the adjacent MBRs are computed as in Table  ST (MDC) = {Tj, Rj | Similar(Ri, R) Λ Adj(Ri, Rj) Λ ( ,  ≥ ε)} T0-Tn

12. Application: The Weather Information and Tactical Support System (WITS) Objective: Development of a Weather Data Repository for Operational Planning –Need to know long term weather conditions –Detect weather anomalies not predicted by forecast models –Integration with External Knowledge Bases Development of an OLAP Weather Repository –Sources: Georgia Weather (1981-2002) US National Weather Service, Georgia Environmental Network, ASOS Modular Development of WITS –Ad-hoc Querying (IQ) –Real time Analysis and Planning (TAPS) Effects on Operational Systems Integration with External Knowledge Bases –Data Mining (DM)

13. WITS System Design

14. WITS: Information Query (IQ) Module for Spatio- Temporal Ad-Hoc Querying –GUI Driven –Drill Down and Roll Up Capabilities

15. WITS/IQ: Drill Down

16. WITS: Tactical Analysis and Planning (TAPS) Integration with External Knowledge Bases –Understand Weather Effects on Systems –Logistical/Route Planning

17. WITS: Data Mining (DM) OLAP and Data Mining Module to show trends and artifacts in the data Detect local weather anomalies not predicted by weather forecasts Example: Trend Analysis of Winter weather in Georgia

18. WITS/DM

19. Application: Situational Awareness in an Urban Environment Heterogeneous Data Sources –Regional Planning, Census, Crime Statistics –Traffic Cams, Overflights –Weather Data (Real-Time, Historical) –Geographical Information Maps with Attributes Challenges –Multiple Formats, Scales –Co-ordinates –Missing Data

20. Conflation Conflation: integration of data from various data sources into digital maps Multiple steps –Feature Matching –Positional Re-alignment –Attribute Deconfliction Query Support

21. Agent-Based Conflation

22. VideoGIS System

23. VideoGIS System: Crime Data

24. VideoGIS System: Video Tracking 1

25. VideoGIS System: Video Tracking 2

26. VideoGIS System: Query Results

27. Conclusions/Future Work Integration of Spatio-Temporal Data, with Multi- Media is challenging Several areas of theoretical development of in –Basic OLAP operations (approximations rollup, drill down) –Query Models Practical Applications in Earth Science and Scientific Data Management