1. Virtual Geophysics Laboratory Scientific workflows exploiting the cloud Ryan Fraser, Terry Rankine, Lesley Wyborn, Joshua Vote, Ben Evans... Presented by Robert Woodcock October 2012 CSIRO | MINERALS DOWN UNDER FLAGSHIP

2. Gather data, process it, publish results Simple, isn’t it? bedrock surficial mineral geochemical geochronologic hyrdrogeological Geo-information geophysical knowledge data

3. Recognise the complete picture....

4. Virtual Geophysics Laboratory | Robert Woodcock 4 | Introducing The Virtual Geophysics Laboratory

5. Data discovery Virtual Geophysics Laboratory | Robert Woodcock 5 | Layers discovered via remote registries Layers consist of numerous remote data services

6. Data discovery Virtual Geophysics Laboratory | Robert Woodcock 6 | Some data services support subsetting Some data services support reformatting e.g. CSV, NetCDF, GeoTIFF

7. Data discovery Virtual Geophysics Laboratory | Robert Woodcock 7 | Some data is only registered with flat files

8. Powered by the Spatial Information Services Stack Common Platform MarineEnvironment, Water Groundwater Geology Geophysics

9. What just happened? Virtual Geophysics Laboratory | Robert Woodcock 9 |

10. Data processing Virtual Geophysics Laboratory | Robert Woodcock 10 | A variety of different scientific codes are already available in the form of “Toolboxes”

11. Data processing Virtual Geophysics Laboratory | Robert Woodcock 11 | Further input files can be uploaded. Input files are passed directly into the cloud

12. Data processing Virtual Geophysics Laboratory | Robert Woodcock 12 | The steps so far have been building an environment to run a processing script...or build from existing templates Either write your own...

13. What just happened? Virtual Geophysics Laboratory | Robert Woodcock 13 | Processing script/ small input files uploaded Start processing Download big data sets Perform data processing Download Job Script/user input files Upload processing results

14. Managing results - provenance Presentation title | Presenter name 14 | All of a job’s outputs are also accessible Each job has a lifecycle that can be managed Successful jobs can have their entire process captured in a ISO 19115 ‘provenance record’

15. What just happened? Virtual Geophysics Laboratory | Robert Woodcock 15 | What’s the processing status? What are the job input/outputs? Publish the job’s process and results Cloud storage will persist the final artefacts

16. Still under construction Virtual Geophysics Laboratory | Robert Woodcock 16 | Courtesy - www.textfiles.com

17. What’s left? BYO cloud allocation Users should be able to authorise VGL start jobs using their compute/storage resources. Confidential Data How do you get access to ‘restricted data’ in a secure manner? Where can you store the results? (geographical restrictions) Massive Horizontal Scaling What’s the best way to set up a truly elastic pool of CPU’s for jobs to utilise? A Common Processing Services Platform – SISS like? Virtual Geophysics Laboratory | Robert Woodcock 17 |

18. Sustainable Resources Policy Societal Need Virtual Solid Earth Sciences Laboratory Environment V. Lab Integrated Virtual Labs Virtual Geophysical Laboratory Virtual Core Laboratory Virtual Geodesy Laboratory Virtual Climate Laboratory Virtual Water Laboratory Virtual Laboratories Geophysics Borehole data Geodesy Climate Modelling Water Monitoring Virtual Libraries Processing Services Data Middleware Processing Services Data Middleware Processing Services Data Middleware Processing Services Data Middleware Processing Services Data Middleware Modelling & analytic tools