1. 1 Implementing Geographic Information System Grid Services Using Distributed Messaging Systems Marlon Pierce Community Grids Lab Indiana University December 15, 2004

2. 2 SERVOGrid Basics SERVOGrid is our project to build a distributed computing infrastructure to support earthquake simulation codes. Distributed Computing Infrastructure Services for remotely executing applications, managing distributed files in logical units, monitoring applications, archiving user projects, and so on. Variously called “Grids” (DOE), “Cyberinfrastructure” (NSF), “e- Science” (UK e-Science program), …. The above is sometimes referred to as an “Execution Grid”. We designed this around a web service architecture. See links at the end for more information. But what about the data? Need ways to access remote data sources through programming interfaces. Practice often is to download (via FTP) entire catalogs and hand edit to get data you want in the format you want. We need to automate this.

3. 3 SERVO Apps and Their Data Disloc: handles multiple arbitrarily dipping dislocations (faults) in an elastic half-space. Relies upon geometric fault models. GeoFEST: Three-dimensional viscoelastic finite element model for calculating nodal displacements and tractions. Allows for realistic fault geometry and characteristics, material properties, and body forces. Relies upon fault models with geometric and material properties. Virtual California: Program to simulate interactions between vertical strike-slip faults using an elastic layer over a viscoelastic half-space. Relies upon fault and fault friction models. RDAHMM: Time series analysis program based on Hidden Markov Modeling. Produces feature vectors and probabilities for transitioning from one class to another. Used to analyze GPS and seismic catalogs.

4. 4 Where Is the Data? QuakeTables Fault Database SERVO’s fault repository for California. Compatible with GeoFEST, Disloc, and VirtualCalifornia http://infogroup.usc.edu:8080/public.html GPS Data sources and formats (RDAHMM and others). JPL: ftp://sideshow.jpl.nasa.gov/pub/mbhftp://sideshow.jpl.nasa.gov/pub/mbh SOPAC: ftp://garner.ucsd.edu/pub/timeseriesftp://garner.ucsd.edu/pub/timeseries USGS: http://pasadena.wr.usgs.gov/scign/Analysis/plotdata/http://pasadena.wr.usgs.gov/scign/Analysis/plotdata/ Seismic Event Data (RDAHMM and others) SCSN: http://www.scec.org/ftp/catalogs/SCSNhttp://www.scec.org/ftp/catalogs/SCSN SCEDC: http://www.scecd.scec.org/ftp/catalogs/SCEC_DChttp://www.scecd.scec.org/ftp/catalogs/SCEC_DC Dinger-Shearer: http://www.scecdc.org/ftp/catalogs/dinger- shearer/dinger-shearer.cataloghttp://www.scecdc.org/ftp/catalogs/dinger- shearer/dinger-shearer.catalog Haukkson: http://www.scecdc.scec.org/ftp/catalogs/hauksson/Socal

5. 5 What Are Web Services? Web Services framework is a way for doing distributed computing with XML. WSDL: Defines interfaces to functions of remote components. This is the programming interface. SOAP: Defines the message format that you exchange between components. This carries the messages over the network. Web Services are not web pages, CGI, servlets, applets. XML provides cross-language support Suitable for both human and application clients. We built SERVOGrid’s execution grid services this way. We are building data grid components in the same fashion. Allows us to build general purpose client environments like Web portals and command line tools. Web Server DB JDBC Browser Web Server SOAP Appl SOAP WSDL

6. 6 User Interface Server DB Service 1 JDBC DB Job Sub/Mon And File Services Operating and Queuing Systems WSDL Browser Interface WSDL Viz Service WSDL Host 1Host 2Host 3 IDL GMT SOAP HTTP(S)

7. 7 GIS Standards for Data Grids Geographic Information Systems ESRI: commercial company with many popular GIS products. Open Geospatial Consortium (formerly OpenGIS Consortium). We will focus on OGC since they define open and interoperable standards. What are the characteristics of a GIS system? Need data models to represent information Need services for remotely accessing data. Need metadata for determining what is stored in the services.

8. 8 Open GIS Services GML abstract data models can encode data but you need services to interact with the remote data. Some example OGC services include Web Feature Service: for retrieving GML encode features, like faults, roads, county boundaries, GPS station locations,…. Web Map Service: for creating maps out of Web Features Sensor Grid Services: for working with streaming, time-stamped data. Problems with OGC services Not (yet) Web Service compliant “Pre” web service, no SOAP or WSDL Use instead HTTP GET/POST conventions. Often define general Web Service services as specialized standards Information services Notification services in sensor grids

9. 9 GML: A Data Model For GIS GML 3.x is a interconnected suite of over 20 connected XML schemas. GML is an abstract model for geography. With GML, you can encode Features: abstract representations of map entities. Geometry: encode abstractly how to represent a feature pictorially. Coordinate reference systems Topology Time, units of measure Observation data.

10. 10 Example Use of GML The SCIGN (Southern California Integrated GPS Network) maintains online catalogs of GPS stations. Collective data for each site is made available through online catalogs. Using various text formats. This is not suitable for processing, but GML is. GML can be used to describe GPS using Feature.xsd schema, with values encoded at GPS observations. www.crisisgrid.org.

11. 11 Anatomy of WFS (G. Aydin) WFS provides three major services as described in OGC specification: GetCapabilities: The clients (WMS servers or users) starts with requesting a document from WFS which describes it’s abilities. When a getCapabilities request arrives, the server dynamically creates a capabilities document and returns this. This is OGC’s formalization of metadata, so important to GEON, ESG, etc. DescribeFeatureType: After the client receives the capabilities document he/she can request a more detailed description for any of the features listed in the WFS capabilities document. The WFS returns an XML schema that describes the requested feature. Metadata about a specific entry. GetFeature: The client can ask the WFS to return a particular portion of any feature data. GetFeature requests contain some property names of the feature and a Filter element to describe the query. The WFS extracts the query and bounding box from the filter and queries the feature databases. The results obtained from the DB query are converted the feature’s GML format and returned to the client as a FeatureCollection.

12. 12 Example WFS Capability Entries ELEMENT NAMEDESCRIPTION Name A name the service provider assigns to the web feature service instance. Title Human-readable title to briefly identify this server in menus. Abstract Descriptive narrative for more information about the server. Keyword Contains short words to aid catalog searching. OnlineResource Defines the top-level HTTP URL of this service. Typically the URL of a "home page" for the service. Fees Contains a text block indicating any fees imposed by the service provider for usage of the service or for data retrieved from the WFS. The keyword NONE is reserved to mean no fees. AccessConstraints Text block describing any access constraints imposed by the service provider on the WFS or data retrieved from that service. The keyword NONE is reserved to indicate no access constraints are imposed.

13. 13 Sample Feature- CA Fault Lines Northridge2 Wald D. J. -118.72,34.243 -118.591,34.176 After receiving getFeature request, WFS decodes this request, creates a DB query from it and queries the database. WFS then retrieves the features from the database and converts them into GML documents. Each feature instance is wrapped as a gml:featureMember element. WFS returns a wfs:FeatureCollection document which includes all featureMembers returned in the query result.

14. A WFS can serve multiple feature types data. WFS returns the results of GetFeature requests as GML documents (Feature Collections). Clients may include other services as well as humans.

15. 15 Conclusion and Other Topics We have reviewed Community Grids Lab efforts to build a GIS-compatible set of data grid services. This provides a unified architecture for grid execution and data grid services. Two other important topics: Managing messaging between components. NaradaBrokering project led by Dr. Shrideep Pallickara. www.naradabrokering.org Managing client environments Component-based portals: www.collab-ogce.org.www.collab-ogce.org Command line and scripting interfaces for web services: www.hpsearch.org. www.hpsearch.org

16. 16 More Information My email: mpierce@cs.indiana.edu Overview (submitted to ACES special issue of PAGEOPH). http://www.servogrid.org/slide/GEM/SERVO/ISERVO_ACES_PAGEOPH.doc QuakeSim Project Page: http://quakesim.jpl.nasa.gov/ QuakeSim Portal: http://www.complexity.ucs.indiana.edu:8282.http://www.complexity.ucs.indiana.edu:8282 QuakeTables Fault Database: http://infogroup.usc.edu:8080/public.html Community Grids Lab GIS Grid Development: http://www.crisisgrid.org/.http://www.crisisgrid.org/ See particularly http://www.crisisgrid.org/html/servo.html for GML information.http://www.crisisgrid.org/html/servo.html See http://www.crisisgrid.org/html/wfs.html for more on Web Feature Service.http://www.crisisgrid.org/html/wfs.html WMS and Information Services information coming soon.