TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview Digital Library Holdings Recent Developments Oceans & Climate Digital Library Portal Australian National Grid Program Enhancing access to and use of digital library data

TPAC Digital Library Who is TPAC? Tasmanian Partnership for Advanced Computing State based APAC (Australian Partnership for Advanced Computing) Located at University of Tasmania, Hobart Partnership between: University of TasmaniaCSIRO Atmospheric Research CSIRO Marine ResearchAustralian Maritime College Australian Antarctic DivisionAntarctic Climate & Ecosystems CRC Bureau of Met. Research Centre

TPAC Digital Library Who is TPAC? Expertise centre for modelling oceans & atmospheres 1) Computational Tools and Techniques Program Aims to develop numerical models for high performance computers with reference to the oceans, atmosphere, Antarctic ice sheet & the environment 2) Education, Outreach and Training Program Aims to establish courses in the application of high performance computer modelling and visualisation of computer models 3) Grid Program Aims to create the tools and facilities that enable digital library data to be made transparently available to users of the APAC National Grid, and to create compute grid applications relevant to earth system science

TPAC Digital Library Architecture Distributed library of oceanographic and climate data Built on OPeNDAP framework - Open-source Project for a Network Data Access Protocol - Makes local data accessible to remote locations over the web (uses web-based transport protocols – http) - Supports a variety of self-describing formats (netCDF, HDF, JGOFS, Matlab, etc) - users can access data of interest (sub-sampling) Accessible using web browsers & common analysis applications (Matlab, Ferret, GrADS, ODC, Excel, etc). Eg. mydata = loaddap(“

TPAC Digital Library OPeNDAP Network APAC NF (Canberra) International IPCC model results (10-50Tb) TPAC 1/8 degree ocean simulations (7Tb) Met Bureau Research Centre (Melbourne) Near real-time LAPS analyses products (<1Gb) Sea- and sub-surface temperature products TPAC & ACE CRC (Hobart) NCEP2 (150Gb), WOCE3 Global (90Gb) Antarctic AWS (150Gb), Climate modelling (4Gb) Sea-ice simulations, CSIRO Marine Research (Hobart) Ocean colour products & climatologies (1Tb) Satellite altimetry data (<1Gb) Sea-surface temperature product CSIRO HPSC (Melbourne) IPCC CSIRO Mk3 model results (6Tb) AC3 Facility (Sydney) Land surface datasets

TPAC Digital Library Architecture

TPAC Digital Library Oceans & Climate Portal

Front-end set of GridSphere portlets – navigational, administrative, and help common and easy to navigate format functionality for searching file meta-data Data API relational database + Java class interfaces Search Crawler console based Java application crawls over OPeNDAP web sites traverses hyperlinks to find OPeNDAP files automatically populates the database TPAC Digital Library Oceans & Climate Portal

TPAC Digital Library Oceans & Climate Portal

TPAC Digital Library Oceans & Climate Portal

TPAC Digital Library Oceans & Climate Portal

TPAC Digital Library Oceans & Climate Portal

Stage 1 complete Stage 2 in progress … - improvements to user interface & search capabilities - provide accounting & authorisation functionality - optimise the back-end database performance - provide support for APAC grid-enabled certificates TPAC Digital Library Oceans & Climate Portal

The APAC Grid program is building a national grid infrastructure integrating the APAC and partner facilities to give researchers seamless access to computational and data resources Also provide services to support research collaboration Activity aligned along two streams – infrastructure and applications 3 infrastructure projects will provide services computing, information, user interface & visualisation 6 scientific areas have been chosen for applications astronomy, bioinformatics, chemistry, earth system science, geoscience and high-energy physics. TPAC activities fall within the earth systems science area TPAC Digital Library APAC National Grid

TPAC Digital Library TPAC Grid Program Data Grid Creating the tools and facilities that enable digital library data to be made transparently available to users of the APAC National Grid Compute Grid Creating compute grid applications relevant to earth system science and environment

TPAC Digital Library Data Grid Activities Expanding our digital library data holdings Globus based OPeNDAP for Grid compatibility Development currently stalled Server-side analysis & visualisation tools eg. GrADS Data Servers & Live Access Servers Catalogues of OPeNDAP repositories Portal provides cataloguing and searching capabilities Future versions of OPeNDAP to have inbuilt cataloguing? OPeNDAP-XML interoperability tool XML version of OPeNDAP in development, eventually leading towards access via a web service

TPAC Digital Library Compute Grid Activities The Grid will support three classes of applications:  User written codes Analysis performed on the Grid within problem solving environments such as Matlab/IDL/GrADS/Ferret  Pre-installed monolithic codes Selected Earth System models will be ported to all HP computing platforms that will be participating in the National Grid.  Earth Systems Science analysis toolkit An Earth Systems Science toolkit will be developed which performs a set of standard discrete operations, such as averaging a set of model output fields, flux calculations, etc.

What we have … Data Discovery Service Oceans & Climate Digital Library Portal Working on … Data Analysis Services Earth Systems Science Analysis Toolkit Earth Systems Science Analysis Workflow Portal Compute Services Selected ESS models running on APAC National Grid TPAC Digital Library Grid Services Summary