1. Semantic Geospatial Data Integration and Mining for National Security Ashraful Alam Bhavani Thuraisingham Ganesh Subbiah Latifur Khan University of Texas at Dallas Shashi Shekhar University of Minnesota

2. Geospatial Data Integration: Motivating Scenario Query: “Find movie theaters within 30 miles of 75080” within, near, overlap – Geospatial Operators Theaters, Restaurants – Businesses (Non-Geospatial data) Miles – Distance Unit 75080, Richardson – Geo References Cinemark Movies 10 Radisson Hotel Dallas North- Richardson

3. Key Contributions Query can be handled by Traditional search engine – Google, Yahoo –Not at Semantic level Almost no search engine facilitates finding relevant web services (except Woogle – template matching for web services & no composition) and handle complex queries DAGIS – Discover geospatial semantic web services using OWL-S Service ontology coupled with geospatial domain specific ontology for automatic discovery, dynamic composition and invocation Facilitates semantic matching of functional and non- functional services from various heterogeneous independent data sources.

4. Key Contributions DAGIS DAGIS DAGIS Query Matchmaker Composer Agent Client Browser Web Service Provider A Web Service Provider B Advertise As Semantic Services Automatic Semantic Query Generation by DAGIS Query Agent Semantic Matching using Matchmaker for Functional and QoS Parameters Dynamic on the Fly Composition for Service orchestration using DAGIS Composer Semantic Query generation Web Service Provider Z …

5. OWL-S Upper Ontology Mapping to WSDL communication protocol (RPC, HTTP, …) marshalling/serialization transformation to and from XSD to OWL Control flow of the service Black/Grey/Glass Box view Protocol Specification Abstract Messages Capability specification General features of the Service Quality of Service Classification in Service taxonomies

6. DAGIS System Architecture

7. Generation of Semantic enabled profile for Geospatial Query Query Profile MilesZipCode Theaters Domain Ontology (Snapshot)Generated OWL-S Semantic Profile http://www.utdallas.edu/~gxs059000/Query.owlhttp://www.utdallas.edu/~gxs059000/OGCServiceontology.owl in OWL

8. Geospatial Service Selection and Discovery DAGIS Agent OWL-S MX Matchmaker 1 Best Service Match : Functionality,QoS Degrees of Match: EXACT < PLUG-IN < SUBSUMES< SUBSUMED-BY<LOGIC BASED FAIL < NEAREST-NEIGHBOUR < FAIL 1 OWLS-MX has been developed by Benedikt Fries and Matthias Klusch at the German Research Center forGerman Research Center for Artificial Intelligence (DFKI Saarbruecken, Germany)

9. Geospatial Service Invocation -OWL-S grounding -WSDL Grounding -Service Invocation through AXIS GetTheater Process MilesZipCode Theaters GetTheater Atomic Process

10. DAGIS System Flow DAGIS Query Interface OWL-S MatchMaker OWL-DL Reasoner for Matchmaker 1 Service Providers 1. Register/ Advertise 3. Service Discovery, Service Enactment DAGIS Matchmaker Service Provider - 1 Service Provider - 1 Service Provider - n Service Provider - n DAGIS Agent DAGIS Agent Reasoner/ Matching Engine Reasoner/ Matching Engine DAGIS Interface DAGIS Interface … 2. Query 1 Pellet is an open source, OWL DL reasoner: http://pellet.owldl.com/

11. DAGIS for Complex Queries 1. Query Profile 2. Service Discovery 3. Compose Selection 4. Construct Sequence 5.Return Dynamic Service URI DAGIS Composer DAGIS Composer Match- Maker Match- Maker DAGIS Agent DAGIS Agent Client Composer Sequencer Composer Sequencer Find Movie Theaters within 30 Miles from Richardson, TX TX Zipcode Finder Zipcode Finder Theater Finder Theater Finder Richardson 30 Miles Theaters 6. Service Invocation

12. DAGIS Composer Algorithm Recursive Back Chaining Inference Mechanism (Regression Planning) TX Richardson 30 Miles Movie Theaters Zipcodefinde r GetTheater Inputs:= City, State, DistanceOutput := Movie Theaters NO Service Provider Inputs:= City, State Output := ZipCode ZipCodeFinder Inputs:= ZipCode, DistanceOutput := MovieTheatersTheater Finder

13. DAGIS Query Interface

14. DAGIS Integration Scenarios QueryAvailabilityService TypeService Invoked Find Movie Theaters within 30 Miles of 75080 YESAtomic (Single) Service Provider GetTheatersAndMoviesS ervice Find Movie Theaters within 30 Miles of Richardson,TX NODAGIS Composes two Atomic Services: ZipCodeFinder, GetTheatersAndMovies Find Movie Theaters within 30 Miles of 75080 QoS: Response Time 30 Sec YES Two services with QoS 40 Sec and 50 Sec Available Atomic Service GetTheatersAndMoviesS ervice QoS Response Time 40 Sec Find Movie Theaters within 30 Miles of 75080 NOAtomic ServiceGetTheatersService

15. Geospatial Data Mining: Case Study: Dataset ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) To obtain detailed maps of land surface temperature, reflectivity and elevation. ASTER obtains high-resolution (15 to 90 square meters per pixel) images of the Earth in 14 different wavelengths of the electromagnetic spectrum, ranging from visible to thermal infrared light. ASTER data is used to create detailed maps of land surface temperature, emissivity, reflectivity, and elevation.

16. Case Study: Dataset & Features Remote sensing data used in this study is ASTER image acquired on 31 December 2005. Covers northern part of Dallas with Dallas-Fort Worth International Airport located in southwest of the image. ASTER data has 14 channels from visible through the thermal infrared regions of the electromagnetic spectrum, providing detailed information on surface temperature, emissive, reflectance, and elevation. ASTER is comprised of the following three radiometers : Visible and Near Infrared Radiometer (VNIR --band 1 through band 3) has a wavelength range from 0.56~0.86μm.

17. Case Study: Dataset & Features Short Wavelength Infrared Radiometer (SWIR-- band 4 through band 9) has a wavelength range from 1.60~2.43μm. Mid-infrared regions. Used to extract surface features. Thermal Infrared Radiometer (TIR --band 10 through band 14) covers from 8.125~11.65μm. Important when research focuses on heat such as identifying mineral resources and observing atmospheric condition by taking advantage of their thermal infrared characteristics.

18. ASTER Dataset: Technical Challenges Testing will be done based on pixels Goal: Region-based classification and identify high level concepts Solution Grouping adjacent pixels that belong to same class Identify high level concepts using ontology- based mining

19. Sketches: Process of Our Approach ASTER Image Training Data Features (14/pixel) SVM Classifiers Test Data Features (14/pixel) All Pixel Data Features (14/pixel) Feature Extraction Feature Extraction Feature Extraction Classifier Training Validation Classification High Level Concepts Pixel Grouping

20. SVM Classifiers: Atomic Concepts Classes Train set Test set Water 1175 1898 Barren Lands 1005 1617 Grass 952 1331 Trees 887 1479 Buildings 1041 768 Road435 648 House15841364 # of instances70799105 Different Class Distribution of Training and Test Sets

21. Process of Our Approach Testing Image Pixels SVM Classifier Pixel Merging Ontology Driven Mining Classified Pixels Concepts and Classes High Level Concepts Training Image Pixels

22. Ontology-Driven Mining Ontology will be represented as a directed acyclic graph (DAG). Each node in DAG represents a concept Interrelationships are represented by labeled arcs/links. Various kinds of interrelationships are used to create an ontology such as specialization (Is-a), instantiation (Instance-of), and component membership (Part-of). Residential Apartment Single Family Home Multi-family Home IS-A Urban Part-of

23. Ontology-Driven Mining We have developed domain-dependent ontologies Provide for specification of fine grained concepts Concept, “Residential Area” can be further categorized into concepts, “House”, “Grass” and “Tree” etc. Generic ontologies provide concepts in coarser grain

24. Ontology Driven Mining Urban AreaResidential AreaOpen Area Target Area BuildingHouseWater Barren Land RoadGrass Tree

25. Challenges Region growing Find out regions of the same class Find out neighboring regions Merge neighboring regions Not scalable Irregular regions Of different sizes Hard to track boundaries or neighboring regions Pixel merging Only neighboring pixels considered Pixels are converted into Concepts Linear

26. Pixels Merging

27. Complexity There are two iterations: First iteration converts signature classes into Concepts Second iteration converts remaining classes and isolated concepts into Dominating classes Each pixels take O(1) time Target area takes O(n) time, where n is the number of pixels in the target area Example (next slide): Signature classes: c1, c2, c3 Non-signature class: c4 Concepts: C1, C2, C3

28. Implementation Software: ArcGIS 9.1 software. For programming, we use Visual Basic 6.0 embedded in the software.

29. Output:

30. Directions Research in Geospatial data integration, mining and security us funded by Raytheon Corporation Stared a joint project with Prof. Shashi Shekhar, University of Minnesota on Spatio-Temporal Data Mining for Crime Analysis funded by NGA (National Geospatial Intelligence Agency)

31. Spatio-Temporal Pattern Mining for Multi-Jurisdiction Multi-Timeframe (MJMT) Activity Datasets Investigators: Shashi Shekhar,(U Minnesota) Bhavani T., L. Khan(U.T.Dallas) Funding Agency: NGA  Motivation: Many Applications  Example: Urban Crime patterns, Sensor Data, …  Pattern Families: Hotspots, Journey to crime, trends, …  Tasks: Crime Prevention, Patrol routes/schedule, …  Problem Definition  Inputs: (i) Activity reports with location and time (ii) Pattern families  Output: Pattern instances  Objective Function: Accuracy, Scalability  Constraints: Urban transportation network

32. Challenge 1: Spatio-Temporal (ST) Nature of Patterns State of the Art: Environmental Criminology Spatial Methods: Hotspots, Spatial Regression Space-time interaction (Knox test) Critical Barriers: richer ST semantics Ex. Trends, periodicity, displacement Approach: 1.Categorize pattern families 2.Quantify: interest measures 3.Design scalable algorithms 4.Evaluate with crime datasets Challenges: Trade-off b/w Semantic richness and Scalable algorithms High activity: 2300 -0000 hrs Rings = weekdays; Slices = hour (Source: US Army ERDC, TEC)

33. 2: Activites on Urban Infrastructure ST Networks State of the Art: Environmental Criminology Largely geometric Methods Few Network Methods: Journey to Crime (J2C) Critical Barriers: Scale: Houston – 100,000 crimes / year Network based explanation Spatio-temporal networks Approaches: 1.Scalable algorithms for J2C analysis 2.Network based explanatory models 3.Time-aggregated graphs (TAG) Challenges: Key assumptions violated! Ex. Prefix optimality of shortest paths Can’t use Dijktra’s, A*, etc. (b) Output: Journey- to-Crime (thickness = route popularity) Source: Crimestat ( a) Input: Pink lines connect crime location & criminal’s residence

34. 3: Multi-Jurisdiction Multi-Temporal (MJMT) Data State of the Art: Spatial, ST ontologies Few network ontologies Critical Barriers: Heterogeneity across networks Uncertainty – map accuracy, gps, … Approach: 1.Ontologies: ST Network activities 2.Integration methods: MJMT data 3.Location accuracy models Challenges: Test datasets Evaluation methods N1 N2N3N4N5 R1R2 R3 Transition Edge Subway Stations Road Intersections