AuScope Grid: ready to take the data plunge Bob Haydon AuScope On behalf of many partners

What is the AuScope Grid? AuScope Grid = Open standards based web services infrastructure 4 year project to build the IT infrastructure a suite of IT tools for spatial data interoperability Core ingredient is an information exchange layer that allows “Clients” to connect to “Resources” such as an organisations data Forms a key component of the AuScope Infrastructure System

Simple Message A large group of research collaborators have worked on the Grid challenge for a decade Aimed at rectifying the national issue of information access, interoperability and cross-organisational collaboration A solution has been developed and Australia is well positioned to lead from here A long road ahead but we have some real and relevant examples of the infrastructure adding value to government organisations The value will be ever so much greater with involvement by industry end users who rely on the data and information

AuScope and the Grid partnership Improving our science base through research is heavily dependent on science infrastructure Physical Equipment Data Software IT Infrastructure Much of this is relies on collaboration AuScope Research Partners State and Commonwealth Government CGGC and CSIRO AuScope Grid Project lead by CSIRO

We rely on data for decision making ? Commonwealth State Local Regional Industry Research bedrock surficial mineral geochemical geochronologic hyrdrogeological Geo-information geophysical We know we all rely on data for our decision making And in relation to Geo-information, the decision makes are from layers of government, Academia and Industry But it’s no longer about using these systems to get data or produce a simple map display –its about analysing and using the data and information i.e “about helping people do stuff” HOWEVER THERE HAAS BEEN A NATIONAL ISSUE OF INFORMATION ACCESS, INTEROPERABILITY AND CROSS-ORGANISATIONAL COLLABORATION. Data Decision makers .... there has been a national issue of information access, interoperability and cross-organisational collaboration

The Grid Challenge over the past decade To rectify the national issue of information access, interoperability and cross-organisational collaboration Develop a suite of IT tools that permits users across government agencies and research organisations to access spatial data in public archives Build upon existing open source Information Technologies where possible Provide web-based tools to discover what data and information exists and to access it, use it and if necessary down load it Develop a system sufficiently robust to deploy to a range of national organisations in a way that encourages other end users to utilise the services An Ambitious Objective The Grid Challenge was to rectify this issue

Collaborators Geological Surveys through CGGC and Government Geoscience Information Committee Collaborators Universities ANU/RSES Macquarie University UWA Adelaide University Curtin University Melbourne University Monash University Sydney University University of Tasmania University of Queensland Federal Government Agencies and Major Initiatives State Government Agencies for Land Management Qld NRM NSW Dept Lands Tas DPIW WA Landgate Other State Government Agencies participate through: Geoscience Australia Even More Ambitious when we look at the range of collaborators!

Collaboration Required Persistence and Endurance Large number of diverse organisations Massive amounts of historical data and information New IT infrastructure was needed capable of providing new data streams Users with multitude of workflows Large range of software applications Different Organisational IT Infrastructure

The infrastructure conceived a decade ago is changing... Increasing need for: richer content diverse content from multiple sources How do I connect.... my tools to your content? AuScope’s Infrastructure vision was an exemplar to help people do this a bit easier

AuScope Infrastructure System for National Data and Integration a combination of research infrastructure and applied science infrastructure The AuScope Infrastructure was in a way an exemplar of an infrastructure system for National Data and Integration. AuScope is an infrastructure system which is a combination of research infrastructure and applied science infrastructure We have physical equipment and data at the back end A National Spatial context for the data Grid Infrastructure to provide access and interoperability S&M provides a toolkit of modelling and simulation software All to help develop testable concepts and models Grid is also building a series of Web Portals for access to data, information and analysis tools to facilitate building Earth Models for the Australian Continent Models and concepts for the Australian Continent Physical equipment and datasets National spatial framework Grid computing infrastructure to access data Software for analysis and modelling 10

The familiar data interfaces ≠ Vision Tas ACT Data Providers Vic That was the vision but what was happening? Most organizations already have web tools that allow users to access their data. These have been built over many years: Quite user friendly Designed to meet specific client requirements Far more functionality However, this does not exactly match the Vision of a Nationally integrated IT infrastructure because the User must find the Provider and use the Provider’s client application NT Provider defined clients and formats Users

The real story was reflected in hard facts Up to 80% of project time is spent compiling and manipulating existing data (example from GSV) 20% of project time available for real work Effective data integration remained elusive Information access and exchange issues plagued the minerals industry and Government (as reported in the Minerals Exploration Action Agenda) and significantly impacted many minerals related research objectives which often require nationally integrated datasets. Cost of data integration is itself a barrier to the research required to justify the investment in data integration True for geology but pervades the entire resource value chain The reality could be seen from some hard facts: ...There was a driver to change and ... ... an opportunity through the AuScope investment

Two projects addressing national data infrastructure and integration AuScope Grid: Spatial Information Services Stack National data sharing infrastructure AuScope National Virtual Core Library Hyperspectral mineral mapping There were two projects conceived and lead by CSIRO that were part of the AuScope program. Underpinned by Australia’s Global leadership in pre-competitive geoscience data 13

AuScope Community Agreements A solution: Open standards based earth science information infrastructure Client AuScope Community Agreements A solution could be seen by adopting and developing an open standards based earth science information infrastructure. What If we could coordinate organizational behavior at the boundaries between organizations? + The point of truth would be corrected if an error is found for everyone that follows + Translating across vocabularies wouldn’t be left to assumption + There would be operational readiness for new events or opportunities that demand a different workflow across organisations + The cost of data integration would lower across the innovation system

Australia is well positioned to lead… Strong industry State and Territory Surveys and GA Precompetitive data and knowledge Exploration initiatives Strong and active Chief Government Geologists Committee World class research capability Track record DET CRC; CET; Universities Supportive Ministerial Council for Minerals and Petroleum Resources Issues recognised and reported in the Minerals Exploration Action Agenda However, Australia is well-positioned to lead A national competitive advantage

Some excellent progress has been made on a long road NVCL – hyperspectral data from core Earth Resources – mineral occurrence data CSIRO MDU Earth model Historical Reports Yilgarn Geochemistry National Geophysics - grav/mag/rads grids and data WA CoE 3D Mineral Mapping BOM Weather station data Coastal Biophysical Modeller

Examples CSIRO MDU and BOM

Discovery Portal – combine and visualise data in different formats from different hosts You can combine various kind of data, stored at different places and in different formats and visualise them in your web browser.

AuScope National Virtual Core Library http://portal.auscope.org/gmap.html A major opportunity understand ore systems Online, Content searchable, and open standard delivery back to your application. http://nvcl.csiro.au 19

Unlocking National Resource Value will involve ongoing collaboration : resembles a national JV National surveys / GA A national ‘JV’ Unlocking National Resource Value So we are probably somewhere in this triangle but need more engagement and adoption to take us to another level. Exploration is a knowledge business! Explorers Innovation & education

The future An Australian Geoscience portal More earth resources-related datasets Broader contribution of data supported by industry users An e-Research infrastructure to provide data management and assimilation tools to integrate, synthesise and access information and knowledge from these large and distributed data stores infrastructure to meet future demand by enhancing multi-dimensional 3/4D data access via the web portals a framework that will allow online access to larger volume data sets and enable processing at their highest resolution with improved model fidelity.

Conclusions Federated Spatial Information Services data infrastructure is now a reality The community has an opportunity to embrace these developments We have only scratched the surface Not just added another dataset or portal to the science community, but provided a new way of doing science, a true infrastructure development We can now transform the way that data and data products are identified, shared, integrated, and reused, to unlock the benefits of true integration of research efforts across the minerals value chain