“How to Connect Campus Grids to the NGS” Neil Geddes Director, GOSC The UK's National Grid Service is a project to deploy and operate a grid infrastructure for computing and data access across the UK. This development will be a cornerstone of the development of the UK's "e- Infrastructure" over the coming decade. The goals, current status and plans for the National Grid Service and the Operations Support Centre will be described.

Outline Overview of GOSC and NGS Services & Getting Access Joining the NGS –How and why The Future –Roadmap for the future Summary

GOSC The Grid Operations Support Centre is a distributed “virtual centre” providing deployment and operations support for the NGS and the wider UK e-Science programme. - started October 2004

GOSC Roles  UK Grid Services  National Services  Authentication, authorization, certificate management, VO management, security, network monitoring, help desk + support centre.   NGS Services  Job submission, simple registry, data transfer, data access and integration, resource brokering, monitoring and accounting, grid management services, workflow, notification, operations centre.  NGS core-node Services  CPU, (meta-) data storage, key software  Services coordinated with others (eg OMII, NeSC, LCG, EGEE):  Integration testing, compatibility & Validation Tests, User Management, training  Administration:  Security  Policies and acceptable use conditions  SLA’s, SLD’s  Coordinate deployment and Operations

The National Grid Service: Towards the UK's e-infrastructure The UK's National Grid Service is a project to deploy and operate a grid infrastructure for computing and data access across the UK. –Learn what it means –Learn how to do it This development will be a cornerstone of the development of the UK's "e-Infrastructure" over the coming decade

NGS “Today” Projects e-Minerals e-Materials Orbital Dynamics of Galaxies Bioinformatics (using BLAST) GEODISE project UKQCD Singlet meson project Census data analysis MIAKT project e-HTPX project. RealityGrid (chemistry) Users Leeds Oxford UCL Cardiff Southampton Imperial Liverpool Sheffield Cambridge Edinburgh QUB BBSRC CCLRC. Nottingham … Interfaces OGSI::Lite If you need something else, please say!

Country providing resources Country anticipating joining In EGEE:  113 sites, 30 countries  >10,000 cpu  ~5 PB storage Includes non-EGEE sites: 9 countries 18 sites EGEE Resources: Feb 2005

Enabling Grids for E-sciencE INFSO-RI GOSC Management Board - NGS StatusEGEE Third Conference, Athens, Applications HEP Applications Biomed Applications –imaging, drug discover –mri simulation –protein sequence analyis Generic Applications –Earth Observation, Seismology, Hydrology, Climate, Geosciences –Computational Chemistry –Astrophysics Applications “behind the corner” –R-DIG –BioDCV

NGS core nodes : Need UK e-Science certificate (1-2 days) Apply through NGS web site (1-2 weeks)

Service and Access What we provide and how to get permission

NGS Core Services Globus Toolkit version 2 Job submission, File transfer, Shell Storage Resource Broker Oracle (9i) OGSA-DAI Certificate Authority Information Services (MDS/GIIS) MyProxy server Integration tests and database Cluster monitoring LCG-VO In testing: VOMS EDG Resource Broker Portal(s)

Gaining Access NGS Partner Sites Data nodes at RAL + Manchester Compute nodes at Oxford + Leeds Compute nodes at Cardiff + Bristol Free at point of use Apply through NGS web site Accept terms and conditions of use Light-weight peer review –1-2 weeks To do: project or VO-based application and registration National HPC services Must apply separately to research councils Digital certificate and Conventional (username/ password) access supported Affiliate Sites Access approved projects/VO’s All access is through digital X.509 certificates From UK e-Science CA or recognized peer

Joining the NGS How, Why and The Vision Thing

How to Join Resource providers join the NGS by Defining level of service commitments through SLDs Adopting NGS acceptable use and security policies Run compatible middleware –as defined by NGS Minimum Software Stack –and verified by compliance test suite Support monitoring and accounting Two levels of membership 1.Affiliation a.k.a. connect to NGS 2.Partnership

How to Join Ask Grid Support Centre, Stephen Pickles Process currently limited only by available effort

How to Join Ask Grid Support Centre, Stephen Pickles Process currently limited only by available effort 2.Get assigned a buddy added to rollout mailing list read the joining guide

How to Join Ask Grid Support Centre, Stephen Pickles Process currently limited only by available effort 2.Get assigned a buddy added to rollout mailing list read the joining guide 3.Install the minimum software stack On the resource, or on a gateway

How to Join Ask Grid Support Centre, Stephen Pickles Process currently limited only by available effort 2.Get assigned a buddy added to rollout mailing list read the joining guide 3.Install the minimum software stack On the resource, or on a gateway 4.Pass monitoring tests for 7 days

How to Join Ask Grid Support Centre, Stephen Pickles Process currently limited only by available effort 2.Get assigned a buddy added to rollout mailing list read the joining guide 3.Install the minimum software stack On the resource, or on a gateway 4.Pass monitoring tests for 7 days 5.Agree to security and acceptable use policies security contact + part of campus security developing operational security with GridPP, EGEE, OSG

How to Join Ask Grid Support Centre, Stephen Pickles Process currently limited only by available effort 2.Get assigned a buddy added to rollout mailing list read the joining guide 3.Install the minimum software stack On the resource, or on a gateway 4.Pass monitoring tests for 7 days 5.Agree to security and acceptable use policies 6.Define a Service Level

How to Join Ask Grid Support Centre, Stephen Pickles Process currently limited only by available effort 2.Get assigned a buddy added to rollout mailing list read the joining guide 3.Install the minimum software stack On the resource, or on a gateway 4.Pass monitoring tests for 7 days 5.Agree to security and acceptable use policies 6.Define a Service Level 7.Set up service level monitoring

How to Join Ask Grid Support Centre, Stephen Pickles Process currently limited only by available effort 2.Get assigned a buddy added to rollout mailing list read the joining guide 3.Install the minimum software stack On the resource, or on a gateway 4.Pass monitoring tests for 7 days 5.Agree to security and acceptable use policies 6.Define a Service Level 7.Set up service level monitoring 8.Approval by GOSC Board –Get representation on Technical Board –Process took 6-12 months for core sites –Process took 3-6 months for Cardiff +Bristol –Process should take less than 3 months for Lancaster

Why the NGS Common tools, procedures and interfaces –Reduce total cost of ownership for providers –Lower threshold for users Early adopter system for UK research grids –technology evaluation (in production) –technology choices –pool expertise –drive interface standards and requirements both a voice and a target –etc.

Why Join (your) Users increasingly want resources as services and not as complicated bits of kit –common interfaces across a range of facilities Funders of regional and national facilities want common interfaces to lower barriers to access –e.g. Hector By joining you leverage the national expertise in running these services –technical advice and support –security procedures and incident response –tools to help monitor and patch All of the above required in any TCO calculation –Get it at lower cost by joining the NGS Members get a say in the technical decisions …

The Future More Vision and Vision Meets Reality

UK e-Infrastructure LHC ISIS TS2 HPCx + HECtoR Users get common access, tools, information, Nationally supported services, through NGS Integrated internationally VRE, VLE, IE Regional and Campus grids Community Grids

Maintaining Compatibility Operating a production grid means valuing robustness and reliability over fashion. –bug uncovered in OpenSSH in October 2004 NGS cares about: –alignment/compatibility with leading international Grid efforts –special requirements of UK e-Science community –easy migration/upgrade paths –proven robustness/reliability –based on standards or standards-track specifications NGS cannot support everything Everyone wants service-oriented grids –but still settling out: WS-I, WS-I+, OGSI, WSRF, GT3, GT4, gLite Caution over OGSI/WSRF has led to wide convergence on GT2 for production grids and hence some inter-Grid compatibility –but there are potentially divergent forces at work Significant changes to NGS Minimum Software Stack will require approval by NGS Management Board on conservative time scales

Strategic Framework GOSC/NGS  UK e-Science project –support other UK (e-)science projects International Compatibility –EGEE European infrastructure (and possible funding) LHC at most UK universities –only user group who want to build the grid –GridPP committed to common w/s plan in 2005 GEANT –Others TeraGrid – US cyberinfrastructure $$$ (unlikely to pay us) Open Science Grid – will develop compatibility with LCG RoW e.g. China –Want use other software, but must be EGEE compatible –Also driven by user requirements –Sets framework for relationship with OMII and others Other factors –JISC and Shibboleth

Process for Moving Forward 1.New developments evaluated by ETF –must have some longer term support likely 2.User requests treated on case by case basis 3.NGS Technical Board consider against needs user demand new functionality improved functionality improved security/performace/managability 4.Proposal brought to GOSC Board Prepared by GOSC “executive” N.Geddes, S.Pickles, A.Richards, S.Newhouse

“User requests treated on case by case basis” Already see users running web services in user space Exactly what we want … but … –Potential security risks Change Conditions of Use to reflect user responsibilities Require secured web services (X509) Encourage auditing and audit trails –With time limits –Services run “at risk” –Services lead to significant system load run on head node (or other specialised node) –Full support only when “approved”

Current Roadmap April 2005Extensions of NGS membership July 2005First assessment of NGS WS infrastructure Outline plans for 2005/2006 NGS AAA devt. Migration and/or interoperability of GT2/GT4 Oct 2005Deployment of Resource Brokering and VO mgmt Client (compute element) compatibility with EGEE Decision on support for GT4 infrastructure Jan 2006Evaluation of Shibboleth interoperability solutions April 2006Approval of Shibboleth interoperability plans July 2006Interoperability with Shibboleth infrastructure Oct 2006 Technology refresh for core NGS nodes

The beginning …

–Backup slides to follow …

© 2004 Electronic Data Systems Corporation. All rights reserved. Step 2: Consolidate (Server, Network, Storage, etc) Step 5: Utility Service Step 1: Migrate & Manage (Regional Facilities) Agility Drivers Standards Visibility Quality Security Efficiency Step 4: Virtual Service Suite Step 3: Automated Operations & Managed Storage Step 6: Grid Reduce Risk Improve Utilisation Improve Scalability, Service Quality/Levels, Productivity & more Reduce TCO EDS is Transforming Clients to Agile Enterprise – Virtualised Computing Platform EDS Services Transition Roadmap

The GOSC Board Director, GOSC (Chair)Neil Geddes Technical Director, GOSCStephen Pickles Collaborating Institutions CCLRCProf. Ken Peach LeedsProf. Peter Dew OxfordProf. Paul Jeffreys ManchesterMr. Terry Hewitt Edinburgh/NeSCProf. Malcolm Atkinson UKERNADr. Bob Day London Collegetbd ETF ChairDr. Stephen Newhouse GridPP Project LeaderProf. Tony Doyle OMII DirectorDr. Alistair Dunlop EGEE UK+I Federation Leader Dr. Robin Middleton HEC LiaisonMr. Hugh Pilcher-Clayton Also invited e-Science User Board Chair. Prof. Jeremy Frey Director, e-Science Core ProgrammeDr. Anne Trefethen