TeraGrid Information Services December 1, 2006 JP Navarro GIG Software Integration

December 1, 2006 TeraGrid Information ServicesPage 2 Outline Motivation Deployed Infrastructure Phase 0 - thru June 2006 Phase 1 - User Portal, Docs, … Phase 2 - Kit Registration Futures - What would you like?

December 1, 2006 TeraGrid Information ServicesPage 3 Motivation The TeraGrid, like any other grid, manages metadata about the organizations, individuals, and infrastructure that make up a grid. Effectively managing this metadata is a critical element of the TeraGrid's infrastructure, software, and services strategy. The TeraGrid ’ s Information Services aims to: Provide standards based grid access to TeraGrid information Accessible thru programmatic interfaces Initially focused on local RP infrastructure metadata Initially based on Globus 4 MDS4 services Information services will be the common Grid interface thru which a resource can publish it ’ s existence and provide information about its capabilities.

December 1, 2006 TeraGrid Information ServicesPage 4 Use Cases / Consumers Meta-schedulers and co-schedulers User portal Gateways TeraGrid documentation Other Grids Applications Application hosting environments

December 1, 2006 TeraGrid Information ServicesPage 5 Deployed Infrastructure Overview mds.teragrid.org RP Resource1 MDS Resource2 MDS Resource3 MDS RP Resource1 MDS RP Resource1 MDS Resource2 MDS Other MDSs WebMDS

December 1, 2006 TeraGrid Information ServicesPage 6 Deployed Infrastructure Details Tooling –MDS4 from CTSS v3 Globus r3 –Is a WSRF service that runs in a GT4 container RP Configuration –RPs responsible for their data –RPs are pre-configured to register with global mds.teragrid.org –Using a combination of push/registration and pull Central (aggregating) Configuration –Aggregating mds.teragrid.org has all the information from all RPs –A public anonymous access service on port –A secure TeraGrid authenticated service on port –WebMDS views of some of the data, follow links at:

December 1, 2006 TeraGrid Information ServicesPage 7 Phase 0 Focused on hypothetical meta-scheduling needs Delivery was targeted for June 2006, when CTSS v3 went production Significant MDS team design and development effort, including new MDS core capabilities to load arbitrary XML data Published data: –queue details –cluster and sub-cluster –file-system –individual compute node details Data sources: –Monitoring data from Nagios, Ganglia, and ClueMon –Local resource managers –Raw XML files for sites not running these 3 monitoring systems Moderate administrator configuration Background material: – –

December 1, 2006 TeraGrid Information ServicesPage 8 Phase 1 Overview –Effort started in June/July 2006 –Phased approach focused on specific real consumers –Lead by DIVS AD Kelly Gaither w/ SI technical assistance –Expected to go production late in Q4 ‘ 06 or early Q1 ‘ 07 Primary focus in delivering information to the User Portal –Scheduling Load –Queue Contents (restricted access) Motivation for Secure/Authenticated MDS4 services Status –Initial information provider deployed at TACC and UC/ANL –Actively in development by user portal –Soon will publish a formal change management document With a TeraGrid wide testing and production plan Other goals for phase 1 –Create information services documentation repository –Engage TeraGrid groups in Metadata and Information Services discussions –Trying to configure some Phase 0 providers to verify they work

December 1, 2006 TeraGrid Information ServicesPage 9 Phase 2 Overview –Effort started in Q –Lead by SI co-AD JP Navarro –Expected to go production mid to late Q1 ‘ 07 Primary focus in delivering CTSS v4 kit registration information Required component of the CTSS v4 Core kit – Actively in development –Mach-up of kit registration –Design document –WebMDS views Will likely include kit specific registration information –Software registration (SoftEnv keys) –Services registration –Other kit specific attributes

December 1, 2006 TeraGrid Information ServicesPage 10 Futures Community Software Registration –Software installed by the community on TeraGrid Resources Community Service Registration –Hosted community services that register themselves –On RP resources that are willing to host services –Application Hosting Environment support Arbitrary XML publishing –Application, user, or job level Key consumers –Gateways –Meta-schedulers, co-schedulers, workflow systems –User portal –User documentation

December 1, 2006 TeraGrid Information ServicesPage 11 Useful links TeraGrid Information Services Page TeraGrid MDS4 Implementation Plan Page TeraGrid MetaData Catalog Metadata, catalogs, and information services list Globus MDS4 documentation