The Solid Earth and Environment Grid Robert Woodcock, CSIRO 6 September 2005

The Solid Earth and Environment Grid Back in 2003… “…It is necessary for open standards and interfaces to be established by communities to be able to interoperate effectively… The Solid Earth and Environment Grid community has been established… to bring together people in the earth, environmental and computing sciences… to address the issues of "transparent access" to data and knowledge about the earth,… and the available and potential technologies offered by the grid… that enhance our ability to explore for and manage our natural and mineral resources.” eXploration and Mining Markup Language (XMML) Gplates ML CoalML Geochemistry Profile Assay Data Exchange Open Geospatial Consortium (OGC) Commission for Geoscience Information Data Model Collaboration Natural Resources Marine Earth Systems NERC DataGrid (UK) CSIRO Exploration and Mining CSIRO Land and Water Tasmanian Partnership for Advanced Computing Australian Antarctic Division British Atmospheric Data Centre (UK) British Oceanographic Data Centre (UK) Council for the Central Laboratory of the Research Councils (UK) Geoscience Australia National Oceans Office Social Change Online University of Sydney Institute of Marine Science Metech Fractal Technologies AusIndustry Minerals Council of Australia Predictive Minerals Discovery CRC National Land and Water Resources Audit Natural Heritage Trust SGI IUGS Commission for Geoscience Information MERIWA WMC Snowden ASIBA Spatial Interoperability Demonstrator Project OGC Australia Mineral Resources Tasmania Minerals Resources New South Wales British Geological Survey (UK) Natural Resources and Mines Queensland Northern Territory Geological Survey Australian Partnership for Advanced Computing BHP Billiton Minerals and Energy Resources South Australia Department of Industry and Resources Western Australia Department of Primary Industries Victoria Placer Dome …. Sincere apologies to any one I missed 228 Registered Users unique visitors per month Over 1000 visits per month Continued steady growth Six sub-web communities SEE Grid Roadmap - Geochemistry WFS online in 3 geological surveys Spatial Interoperability Demonstrator Project APAC Grid for Geosciences Australian-UK collaboration for the Exploitation of Grid and Geospatial Standards. …. OGC WFS: Geoserver OGC WMS Social Change Online - Web Map Composer Metech – Acquire Fractal Technologies – FracSIS Suite Globus Grid Middleware…. 2005

The Solid Earth and Environment Grid The SEE Grid Community Working together (loosely) to develop a toolkit for interoperability for the Solid Earth and Environmental Sciences  Together… because our information and services need to be shared more easily to achieve our goals  Loosely… because ultimately we are separated by political and economic boundaries  Toolkit… because our World is dynamic and we need tools that can be reconfigured and chained together quickly to answer our questions …in this context we must reduce the barriers to becoming a part of the community

The Solid Earth and Environment Grid What is involved in being part of the Community? Interoperability – how? A community shares:  Community Architecture  Information Models  Information Service Interfaces  Computational Service interfaces  Education …SEE Grid seeks to capture and publish these shared aspects What does 2 years of SEE Grid give us? Figures from Software Fortresses by Roger Sessions

The Solid Earth and Environment Grid Architecture Open Geospatial Consortium Notional Architecture  Most projects use OGC NA as a starting point  And its information standards…  And its service interfaces…  It can be quite a stack of paper! Key Lessons:  OGC NA breaks problem into specific aspects  Use it to identify necessary services for your application  Reconcile desire for simplicity with realities of the situation  Formal documentation of assumptions and scope  Write Service Profiles - Check if they exist and use them  It pays to get assistance from specialists in the community – use Roadmaps and Guides  OGC is NOT enough  domain specific profiles required  Many components do NOT support this Figures from Software Fortresses by Roger Sessions

The Solid Earth and Environment Grid Human face of Systems Architecture Nobody understands all the problem domain, technology and institutions… Architecture allows:  Separation of concerns  a business planning capability:  From an architecture one can develop a “roadmap” to address short term compromises  reconcile user needs with provider capabilities  Not everything is there yet…

The Solid Earth and Environment Grid Common Compromises Data isn’t as consistent as you’d hoped  Reduced user functionality  Data remediation/governance improvements  change in business practice! Technology doesn’t quite allow you to jump from what you have to what you need in 1 configurable step…  Components can be improved  SEE Grid attempting to collect and document common requirements to assist community in improving components “Stubbing” of components  Registers in particular though this can’t go on much longer…

The Solid Earth and Environment Grid Information Models Not a storage problem…  User View (Data Products)  Custodian’s View (Consistency, Efficiency)  Systems View (bandwidth, query cost) Semantics and structure  GML, GML application schemas  XMML and other domain specific models – available from version controlled repository, automated validation being setup  Community collaboration addressing cross-domain issues (Earth Systems and Geosciences)  Common patterns emerging – Observations and Measurements  These are fed back to standards bodies From one data store, many different products Specific views are “profiles” of a more complete model  SEE Grid community profiles are starting to be documented

The Solid Earth and Environment Grid Would be nice to have… Architecture  Catalog of Usage and Management Use Cases – this helps identify community issues Information Models  Governance – by whom?  Catalog of Design Patterns – starting to emerge… Education  GML Application Schema Design Tutorial – getting the ball rolling… Profiles = simplicity, consistency  Geochemistry WFS Profile underway, others exist in OGC but many more required in domain specific areas  Document them and place them on SEE Grid Harmonisation  OGC Web Services, SOAP, Grid, WSRF

The Solid Earth and Environment Grid What is expected to be achieved (actually in proposals/plans) National pre-competitive geoscience data grid  Assays – national demonstrator running (upcoming talk)  Hyper-spectral, geophysics, geology, drill holes Full B2B cycle for Assay data exchange Computational Services  Predictive Mineral Discovery – connectivity to Australian Partnership for Advanced Computing Grid for HPC resources. (in progress)  Earthbyte – service chaining (upcoming talk)  Seismic simulation  Mantle convection service (in progress)

The Solid Earth and Environment Grid SRB, ebRIM… Snark Profile, GPML, XMML… EarthBytes Profile, GPML, XMML… ebRIM, WSDL, UDDI… Gridsphere, JSR 168 Portlets… Globus, WS-RF… OGC WFS, XMML, Service Profiles… ebRIM, OGC Catalog Service… The Sample Application… Service Interactions Resource Registry Service Registry Results Archive Data Management Portal Job Management Portal HPC Repository Login Job Monitor Run Simulation Edit Problem Description Local Repository Archive Search Geology S.A Geology W.A Rock Prop. N.S.W Rock Prop. W.A Observational Databases Authorisation, Authentication & Audit Job Management Service Snark Service EarthBytes Service User Workflow... WS-Security, SAML, Shibboleth… Common Component

The Solid Earth and Environment Grid Conclusion “…The Solid Earth and Environment Grid community has been established…” eXploration and Mining Markup Language (XMML) Gplates ML CoalML Geochemistry Profile Assay Data Exchange Open Geospatial Consortium (OGC) Commission for Geoscience Information Data Model Collaboration Natural Resources Marine Earth Systems NERC DataGrid (UK) CSIRO Exploration and Mining CSIRO Land and Water Tasmanian Partnership for Advanced Computing Australian Antarctic Division British Atmospheric Data Centre (UK) British Oceanographic Data Centre (UK) Council for the Central Laboratory of the Research Councils (UK) Geoscience Australia National Oceans Office Social Change Online University of Sydney Institute of Marine Science Metech Fractal Technologies AusIndustry Minerals Council of Australia Predictive Minerals Discovery CRC National Land and Water Resources Audit Natural Heritage Trust SGI IUGS Commission for Geoscience Information MERIWA WMC Snowden ASIBA Spatial Interoperability Demonstrator Project OGC Australia Mineral Resources Tasmania Minerals Resources New South Wales British Geological Survey (UK) Natural Resources and Mines Queensland Northern Territory Geological Survey Australian Partnership for Advanced Computing BHP Billiton Minerals and Energy Resources South Australia Department of Industry and Resources Western Australia Department of Primary Industries Victoria Placer Dome …. Sincere apologies to any one I missed 228 Registered Users unique visitors per month Over 1000 visits per month Continued steady growth Six sub-web communities SEE Grid Roadmap - Geochemistry WFS online in 3 geological surveys Spatial Interoperability Demonstrator Project APAC Grid for Geosciences Australian-UK collaboration for the Exploitation of Grid and Geospatial Standards. …. OGC WFS Social Change Online - Web Map Composer Metech – Acquire Fractal Technologies – FracSIS Suite Geoserver Globus Grid Middleware…. 2005

Thank You CSIRO Exploration and Mining NameRobert Woodcock TitlePrincipal Software Engineer Phone Webwww.seegrid.csiro.au Contact CSIRO Phone Webwww.csiro.au