1. Grid-Enabled Geospatial Systems
OGF-24
Grid-Enabled Geospatial Systems
Craig Lee
Mark Reichardt

2. Geospatial Data: Immense Applicability
Estimates vary, but ~80-90% of all data collected or produced by the human race is geospatially referenced
Image: E. Gennai, C. Terbough, ESRI
Natural exploration, e.g., oil & gas
Public administration
Civic planning and engineering
Weather and aviation
Environmental planning
Disaster management
Satellite ground systems 2

3. Global Earth Observation Grid
An Example: GEO Grid
Global Earth Observation Grid
S. Sekiguchi, AIST 3

4. The OGC-OGF Collaboration
Promote best practices and international standardization for distributed geospatial data processing capabilities that is:
Transparent -- users is not aware of the infrastructure
Interoperable -- the resources work together
Scalable -- small local, to massive distributed platforms
Directly Relevent to Satellite Ground Systems 4

5. Two Issues
What geospatial and computing technologies must be integrated?
How do we manage the process to make progress on doing so? 5

6. Some Key OGC Standards Web Map Service (WMS) Web Feature Service (WFS)
Display of registered and superimposed map-like views of information that come simultaneously from multiple remote and heterogeneous sources.
Web Feature Service (WFS)
Retrieval and update of digital representations of real-world objects referenced to the Earth’s surface.
Web Coverage Service (WCS)
Access to a grid coverage, usually encoded in a binary format, and offered by a server.
Catalogue Service for the Web (CSW)
Common interfaces to publish, discover, browse, and query metadata about data, services, and other resources 6

7. Key OGC Standards Sensor Observation Service (SOS)
Access observations for a sensor or sensor constellation, whether in-situ or dynamic (e.g, water monitoring or satellite imaging). Optionally access associated sensor and platform data
Sensor Planning Service (SPS)
An interface to task sensors or models. Using SPS, sensors can be reprogrammed or reconfigured, sensor missions can be started or changed, simulation models executed and controlled.
Sensor Alert Service (SAS)
Defines an interface for publishing and subscribing to alerts from sensors. If an event occurs the SAS will notify all clients subscribed to this event type.
Observations and Measurements Encoding Standard (O&M)
Defines an abstract model and an XML schema encoding for observations and it provides support for common sampling strategies. It also provides a general framework for systems that deal in technical measurements in science and engineering. 7

8. Key OGC Standards Transducer Markup Language (TML)
TML defines the conceptual model and XML Schema for describing transducers and supporting real-time streaming of data to and from sensor systems
Web Processing Service (WPS)
WPS defines a standardized interface to facilitate the publishing of geospatial processes, and the discovery of an binding to those processes by clients 8

9. OGC Sensor Web Enablement Framework (SWE)
Registered sensor and observation metadata
Metadata for a sensor and observations, and a URL
Metadata for a sensor observations, and a URL
………
CSW
SPS
Mission
Control
System
WNS 
SWE Architecture Encodings:
Observations & Measurements
SensorML
Transducer Markup Language (TML)
SOS
IEE1451
SAS  ! 9

10. OGC Web Services Combined10

11. Key OGF Standards HPC-Basic Profile
Cluster scheduling interoperability layer supported by Microsoft, Altair Computing, Platform Computing, and others
Simple API for Grid Applications (SAGA)
Complete distributed programming environment with C++ and Java bindings
GridRPC
Grid-enabled Remote Procedure Call
Data Access and Integration specs (WS-DAI-*)
Set of specs for accessing remote files, databases, XML documents
Web Services Resource Framework (WSRF)
Allow remote data and services to be independently managed 11

12. An Example of a Global Grid: EGEE
Bob Jones, EGEE Project Director 12

13. Coalescing Service Categories to Specification Areas
Infrastructure Service Categories
Catalogues & Registries
Discovery
Applications
Job Submission/Execution Svcs
Workflow Management
Data Product Management
Storage & Archiving
Messaging
Event Notification
User Management
Information Assurance/Security
Monitor & Control
Enterprise Svc Mgmt/Software Mgmt
APIs/Ops Systems
Financial Accounting
Specification Areas
Catalogues & Registries
Discovery
Applications
Execution Services
Workflow/Transactions
Data Management
State Management
Messaging, Routing, Addressing
Event Notification
Metadata Schemas & Ontologies
Portals and User Interfaces
Security
Policy & Agreement
System Architecture
System Management
APIs Application Program Interfaces
Ops Operations
Svc Service 13

14. Populating the Specification Areas
Applications
(There are essentially an unbounded number of “applications” that could run as part of the application domain. OGC WMS, WFS & WCS are listed here since they would be part of what an end-user sees and interacts with.)
Job Submission/Execution Svcs
WSDL, SOAP, (WS-I Basic Profile), CORBA, JSDL, OGSA-BES (OGF HPC Basic Profile), OGC Web Processing Service
Discovery
UDDI, (WS-I Basic Profile), DDMS v1.3 (DoD Discovery Metadata Specification), WS-Discovery
Catalogues & Registries
LDAP, OpenGIS Cat 2.0, OGC CSW, ebRIM, ebXML RS, EO Products Ext., NATO DFDD (DGIWB Data Feature Data Dictionary), NAS v1.8 (NSG Application Schema), NEC V1.8 (NSG Entity Catalog)
Metadata Schemas & Ontologies
XML, CIM/GLUE, FGDC, GML 3.1, ISO (Dublin Core), ISO (metadata), ISO (services), ISO (sensors), OWL, OWL-S, SWRL, (many other ISO docs)
Messaging, Routing, Addressing
HTTP, WS-Addressing, WS-MessageDelivery, WS-Reliability, WS-ReliableMessaging
Event Notification
WS-Notification, WS-Eventing, Sensor Alert Service
BPEL Business Process Execution Language
CIM Common Information Model
FGDC Federal Geographic Data Committee
FTP File Transfer Protocol
GML Geography Markup Language
HTTP Hypertext Transfer Protocol
ISO International Organization for Standardization
JSDL Job Submission Description Language
LDAP Lightweight Directory Access Protocol
OGC Open Geospatial Consortium
OGSA-BES Open Grid Services Architecture-Basic Execution Services
SAML Security Assertion Markup Language
SOAP Simple Object Access Protocol
SSL/TLS Secure Socket Layer/Transport Layer Security
SWRL Semantic Web Rule Language
UDDI Universal Description, Discovery and Integration
WCS Web Coverage Service
WFS Web Feature Service
WMS Web Map Service
WS Web Services
XACML eXtensible Access Control Markup Language
XML eXtensible Markup Language
Updated from Fox, Ho, Pierce – U. Indiana
Green: On DISR Baseline 14

15. Populating the Specification Areas
Workflow/Transactions
BPEL, WS-Coordination, (Kepler, Triana, Pegasus, …)
Data Management
FTP, GridFTP, OGSA-DAI, (SRB, iRODS)
State Management
IETF RFC 2695 (HTTP State Management (cookies)), WS-ResourceFramework, WS-Context, WS-I+, WS-MetadataExchange
Portals and User Interfaces
WS-RemotePortlets
Security
SSL/TLS, Kerberos, WS-Security, ID-WSF 2.0 (Liberty Identity Web Services Framework), WS-Trust, SAML, GSI, Shibboleth, VOMS (Virtual Organization Management System), OpenID, OAuth
Policy & Agreement
WS-Policy, WS-Agreement, (iRODS), XACML, GeoXACML
System Architecture
OGF Open Grid Services Architecture, OGC Sensor Web Enablement (SWE), Community Sensor Model v2.A (CSM)
System Management
WS-DistributedManagement, WS-Management, WS-Transfer, OGF CDDLM (Configuration Description, Deployment and Lifecycle Management)
BPEL Business Process Execution Language
CIM Common Information Model
FGDC Federal Geographic Data Committee
FTP File Transfer Protocol
GML Geography Markup Language
HTTP Hypertext Transfer Protocol
ISO International Organization for Standardization
JSDL Job Submission Description Language
LDAP Lightweight Directory Access Protocol
OGC Open Geospatial Consortium
OGSA-BES Open Grid Services Architecture-Basic Execution Services
SAML Security Assertion Markup Language
SOAP Simple Object Access Protocol
SSL/TLS Secure Socket Layer/Transport Layer Security
SWRL Semantic Web Rule Language
UDDI Universal Description, Discovery and Integration
WCS Web Coverage Service
WFS Web Feature Service
WMS Web Map Service
WS Web Services
XACML eXtensible Access Control Markup Language
XML eXtensible Markup Language
Updated from Fox, Ho, Pierce – U. Indiana
Green: On DISR Baseline 15

16. How Do We Manage Progress for Distributed Geospatial Systems?
Targeted Projects on Decisive Issues
Build Critical Mass of Key Stakeholders
Continual polling and coordination across the community
They must agree on:
Clear Goals
Clear Schedule (“time-box” the process)
Clear Responsibilities
Properly Provisioning the Effort 16

17. A General Process Model
Concept
Development
Task A
*RFQ/CFP = Request for Quotation/Call for Participation
RFQ/CFP*
Development
Task B
Selection & Kick-off
Task C
Develop & Test
Task D
Clear Schedule, Deliverables and Project Responsibilities
Deploy & Persist
Task E 17 17 17

18. Return on Investment
What are the “carrots” to build the critical mass of stakeholders?
Get early influence in specification development, early skills building, visibility, and opportunity for early market deployment of standards, but just as important…
Return on Investment (ROI)
Investment
Time, Money & People
Both Monetary and In-Kind (labor & materials)
Timely Connection to Concrete Results
Stakeholders benefit from collaboration
Get more than they put in 18

19. A Case Study: HPC Basic Profile
2006
Key stakeholders decided to demonstrate interoperability between their existing job submission infrastructures
November 2006 (SC06)
Prototype implementations demonstrated
28-August-2007
HPC Basic Profile, Version 1.0, published
November 2007 (SC07)
Interoperability demonstrated by Altair, Microsoft, Platform, OMII-UK, OMII-Europe, EGEE, UVa
21-February-2008
Interoperability Experiences with HPCBP, Version 1.0, published
Commercial adoption plans by Altair, Microsoft & Platform 19

20. OGC Web Service Testbed
OWS is an annual process where sponsors identify specific demonstration targets
Larger number of participants supply in-kind resources
OWS-5 recently finished (March 28, 2008)
OWS-5 Sponsors
BAE Systems - National Security Solutions
Federal Geographic Data Committee (FGDC/USGS)
Google
Lockheed Martin Integrated Systems & Solutions (Lead Org.)
Northrop Grumman
US National Aeronautic and Space Administration (NASA)
US National Geospatial-Intelligence Agency (NGA)
~35 Participating Organizations
3x-4x Return on Investment
Strength of Collaboration 20

21. OWS-5 Five Threads Outbrief Videos Sensor Web Enablement (SWE)
Geo Processing Workflow (GPW)
Information Communities and Semantics (ICS)
Agile Geography
Compliance Testing (CITE)
Outbrief Videos
NASA Sensor Web 2.0 Experiments (7:43)
Data Architecture Views (10:37)
Conflation Processing (5:07)
Web Coverage Processing Service (5:58) 21

22. Still from OWS-5 NASA Sensor Web Video
Satellite EO-1 tasked to collect imagery on Northern San Diego County Wildfires that was integrated with UAV track data (red lines) in Google Earth. Demo drove issues around sensor networks, data interoperability, and command & control. 22

23. OWS-6: RFQ/CFP Topics Sensor Web Enablement (SWE) Aviation Information
Georeferenced sensors, event notification, security
Aviation Information
Aeronautical Information Exchange Model (AIXM) for next generation flight control system (FAA and EuroControl)
GeoProcessing Workflows (GPW)
Web Processing Service (WPS), Workflow and Grids (“WPS to Grid”)
Distributed resource management, e.g., data, services, hosts, networks, for geospatial applications
Decision Support Services
3D Visualization with Fly Through
Open Location Services
Integrated Client for multiple OWS services
Compliance, Interoperability & Test Evaluation (CITE)
Suite of tests and reference implementations for all approved standards 23

24. Possible Geospatial/Grid Topics
Integration, interoperability of geospatial and computing resource
Information models (CIM/GLUE)
Distributed and federated catalogs (ebRIM, MDS, etc.)
Discovery services
Implementation of WPS on various grid tools
SAGA, GridRPC, HPC-Basic Profile
Integration of WS-Eventing, WS-Notification, INFOD
Sensor Web Enablement
Integration of configuration and lifecycle management tools
Configuration Description, Deployment and Lifecycle Management (CDDLM)
Integration of WPS with workflow management tools
BPEL, Kepler, Taverna, Triana, Pegasus, etc.
Workflow design tools, execution engines, planning (data virtualization), data provenance
Integration of WPS with grid security models
SAML, XACML, VOMS, GSI
Support for virtual organizations
Integration of geospatial data with rule-based, data management policy engines
iRODS 24

25. Digital Repositories/Libraries
Digital Repositories/Libraries represent a growing part of the “infomass”
Federated catalogues and federated storage
OGF-Europe Workshop at OGF-23
4 sessions, ~70 participants from industry, academia and European Commission
Proposed OGF-Europe collaboration with Global Research Library 2020
Proposed OGF-Europe involvement in GENESI-DR Digital Repositories Seminar in Autumn 2008, Frascati
OGF-Europe PC membership for the DReSNet special DR session at IEEE e-Science conference, Dec 2008
Eventual development of an OGF Working Group dedicated to issues surrounding DRs 25

26. Another Example: Green Grids Managing Energy Policy
Extensive use of virtualization and fiber optics to put the computing where the power is
Enable the workload to “follow the wind” or “follow the sun”
Source:
Bill St. Arnaud 26

27. Another Example: Green Grids Managing Energy Policy
Extensive use of virtualization and fiber optics to put the computing where the power is
Enable the workload to “follow the wind” or “follow the sun”
Source:
Bill St. Arnaud 27

28. Another Example: Green Grids Managing Energy Policy
Extensive use of virtualization and fiber optics to put the computing where the power is
Enable the workload to “follow the wind” or “follow the sun”
A geospatial sensor network
Source:
Bill St. Arnaud 28

29. Summary
Significant work already underway for standard, distributed geospatial computing infrastructures
Highly relevant to many application domains
Opportunity to leverage the work of key participants through collaboration
How to engage application groups and industries needing distributed geospatial processing? 29

30. Back-Ups and Other Draft Material

31. A “Cloudy” Solution Space
Vendors of high-level
application configuration
tools are potential
“cloud framework providers”
map
reduce
Data Collection
OS Virtualization
Amazon S3/EC2
RightScale, GigaSpaces,
Elastra, 3Tera
Cohesive
Hadoop over EC2
Sun’s Caroline
Hadoop
MS Astoria
Mesh
MapReduce, GFS, BigTable
Google AppEngine
Parallel Frameworks
Software as a Service
…..
Courtesy of Dennis Gannon, 31

32. A Host of Cloud Issues Data access and interoperability Security
Will have to be approached at the application domain level, by the domain users involved
Security
Can cloud providers provide sufficient security for sensitive applications?
Reliability
Can cloud providers provide sufficient reliability for critical applications?
Frameworks
How to manage sets of VMs and VOs?
These are essentially “cloud frameworks” and call for "cloud framework providers"
Performance management
Clouds tend to abstract away location
Managing the compute-data locality (affinity) can be important
Costing models
How to compare your own infrastructure costs with a cloud computer?
How to compare two clouds? 32