Presentation on theme: "AuScope Grid: ready to take the data plunge"— Presentation transcript:
1AuScope Grid: ready to take the data plunge Bob HaydonAuScopeOn behalf of many partners
2What is the AuScope Grid? AuScope Grid = Open standards based web services infrastructure4 year project to build the IT infrastructurea suite of IT tools for spatial data interoperabilityCore ingredient is an information exchange layer that allows “Clients” to connect to “Resources” such as an organisations dataForms a key component of the AuScope Infrastructure System
3Simple MessageA large group of research collaborators have worked on the Grid challenge for a decadeAimed at rectifying the national issue of information access, interoperability and cross-organisational collaborationA solution has been developed and Australia is well positioned to lead from hereA long road ahead but we have some real and relevant examples of the infrastructure adding value to government organisationsThe value will be ever so much greater with involvement by industry end users who rely on the data and information
4AuScope and the Grid partnership Improving our science base through research is heavily dependent on science infrastructurePhysical EquipmentDataSoftwareIT InfrastructureMuch of this is relies on collaborationAuScope Research PartnersState and Commonwealth Government CGGC and CSIROAuScope Grid Project lead by CSIRO
5We rely on data for decision making ?CommonwealthStateLocalRegionalIndustryResearchbedrocksurficialmineralgeochemicalgeochronologichyrdrogeologicalGeo-informationgeophysicalWe know we all rely on data for our decision makingAnd in relation to Geo-information, the decision makes are from layers of government, Academia and IndustryBut it’s no longer about using these systems to get data or produce a simple map display –its about analysing and using the data and information i.e “about helping people do stuff”HOWEVER THERE HAAS BEEN A NATIONAL ISSUE OF INFORMATION ACCESS, INTEROPERABILITY AND CROSS-ORGANISATIONAL COLLABORATION.DataDecision makers.... there has been a national issue of information access, interoperability and cross-organisational collaboration
6The Grid Challenge over the past decade To rectify the national issue of information access, interoperability and cross-organisational collaborationDevelop a suite of IT tools that permits users across government agencies and research organisations to access spatial data in public archivesBuild upon existing open source Information Technologies where possibleProvide web-based tools to discover what data and information exists and to access it, use it and if necessary down load itDevelop a system sufficiently robust to deploy to a range of national organisations in a way that encourages other end users to utilise the servicesAn Ambitious ObjectiveThe Grid Challenge was to rectify this issue
7Collaborators Geological Surveys through CGGC and Government GeoscienceInformation CommitteeCollaboratorsUniversitiesANU/RSESMacquarie UniversityUWAAdelaide UniversityCurtin UniversityMelbourne UniversityMonash UniversitySydney UniversityUniversity of TasmaniaUniversity of QueenslandFederal Government Agenciesand Major InitiativesState Government Agenciesfor Land ManagementQld NRMNSW Dept LandsTas DPIWWA LandgateOther State GovernmentAgencies participate through:Geoscience AustraliaEven More Ambitious when we look at the range of collaborators!
8Collaboration Required Persistence and Endurance Large number of diverse organisationsMassive amounts of historical data and informationNew IT infrastructure was needed capable of providing new data streamsUsers with multitude of workflowsLarge range of software applicationsDifferent Organisational IT Infrastructure
9The infrastructure conceived a decade ago is changing... Increasing need for:richer contentdiverse content from multiple sourcesHow do I connect....my tools toyour content?AuScope’s Infrastructure vision was an exemplar to help people do this a bit easier
10AuScope Infrastructure System for National Data and Integration a combination of research infrastructureand applied science infrastructureThe AuScope Infrastructure was in a way an exemplar of an infrastructure system for National Data and Integration.AuScope is an infrastructure system which is a combination of research infrastructure and applied science infrastructureWe have physical equipment and data at the back endA National Spatial context for the dataGrid Infrastructure to provide access and interoperabilityS&M provides a toolkit of modelling and simulation softwareAll to help develop testable concepts and modelsGrid is also building a series of Web Portals for access to data, information and analysis tools to facilitate building Earth Models for the Australian ContinentModels and concepts for the Australian ContinentPhysical equipment and datasetsNational spatial frameworkGrid computing infrastructure to access dataSoftware for analysis and modelling10
11The familiar data interfaces ≠ Vision TasACTData ProvidersVicThat was the vision but what was happening?Most organizations already have web tools that allow users to access their data.These have been built over many years:Quite user friendlyDesigned to meet specific client requirementsFar more functionalityHowever, this does not exactly match the Vision of a Nationally integrated IT infrastructure because the User must find the Provider and use the Provider’s client applicationNTProvider definedclients and formatsUsers
12The real story was reflected in hard facts Up to 80% of project time is spent compiling and manipulating existing data (example from GSV)20% of project time available for real workEffective data integration remained elusiveInformation access and exchange issues plagued the minerals industry and Government (as reported in the Minerals Exploration Action Agenda) and significantly impacted many minerals related research objectives which often require nationally integrated datasets.Cost of data integration is itself a barrier to the research required to justify the investment in data integrationTrue for geology but pervades the entire resource value chainThe reality could be seen from some hard facts:...There was a driver to change and ...... an opportunity through the AuScope investment
13Two projects addressing national data infrastructure and integration AuScope Grid: Spatial Information Services StackNational data sharing infrastructureAuScope National Virtual Core LibraryHyperspectral mineral mappingThere were two projects conceived and lead by CSIRO that were part of the AuScope program.Underpinned by Australia’s Global leadership in pre-competitive geoscience data13
14AuScope Community Agreements A solution: Open standards based earth science information infrastructureClientAuScope Community AgreementsA solution could be seen by adopting and developing an open standards based earth science information infrastructure.What If we could coordinate organizational behavior at the boundaries between organizations?+ The point of truth would be corrected if an error is found for everyone that follows+ Translating across vocabularies wouldn’t be left to assumption+ There would be operational readiness for new events or opportunities that demand a different workflow across organisations+ The cost of data integration would lower across the innovation system
15Australia is well positioned to lead… Strong industryState and Territory Surveys and GAPrecompetitive data and knowledgeExploration initiativesStrong and active Chief Government Geologists CommitteeWorld class research capabilityTrack recordDET CRC; CET; UniversitiesSupportive Ministerial Council for Minerals and Petroleum ResourcesIssues recognised and reported in the Minerals Exploration Action AgendaHowever, Australia is well-positioned to leadA national competitive advantage
16Some excellent progress has been made on a long road NVCL – hyperspectral data from coreEarth Resources – mineral occurrence dataCSIRO MDU Earth modelHistorical ReportsYilgarn GeochemistryNational Geophysics - grav/mag/rads grids and dataWA CoE 3D Mineral MappingBOM Weather station dataCoastal Biophysical Modeller
18Discovery Portal – combine and visualise data in different formats from different hosts You can combine various kind of data, stored at different places and in different formats and visualise them in your web browser.
19AuScope National Virtual Core Library A major opportunity understand ore systemsOnline,Content searchable,andopen standard delivery back to your application.19
20Unlocking National Resource Value will involve ongoing collaboration : resembles a national JV National surveys / GAA national ‘JV’UnlockingNationalResourceValueSo we are probably somewhere in this triangle but need more engagement and adoption to take us to another level.Exploration is a knowledge business!ExplorersInnovation & education
21The future An Australian Geoscience portal More earth resources-related datasetsBroader contribution of data supported by industry usersAn e-Research infrastructure to providedata management and assimilation tools to integrate, synthesise and access information and knowledge from these large and distributed data storesinfrastructure to meet future demand by enhancing multi-dimensional 3/4D data access via the web portalsa framework that will allow online access to larger volume data sets and enable processing at their highest resolution with improved model fidelity.
22ConclusionsFederated Spatial Information Services data infrastructure is now a realityThe community has an opportunity to embrace these developmentsWe have only scratched the surfaceNot just added another dataset or portal to the science community, but provided a new way of doing science, a true infrastructure developmentWe can now transform the way that data and data products are identified, shared, integrated, and reused, to unlock the benefits of true integration of research efforts across the minerals value chain