1. GOSC and NGS Status Neil Geddes Director, GOSC Thanks to Stephen Pickles and Andy Richards http://www.ngs.ac.uk http://www.grid-support.ac.uk

2. Outline Users –registrations, usage, helpdesk queries Services & Middleware –what we offer today –managing change Expansion –“joining” the NGS

3. Users

4. Gaining Access NGS core nodes data nodes at RAL and Manchester compute nodes at Oxford and Leeds free at point of use apply through NGS web site accept terms and conditions of use light-weight peer review –1-2 weeks to do: project or VO-based application and registration all access is through digital X.509 certificates –from UK e-Science CA –or recognized peer National HPC services HPCx CSAR Must apply separately to research councils Digital certificate and Conventional (username/ password) access supported

5. User registrations so far

7. User Survey Query all users who have had accounts > 3 months –16 responses (out of ~100 users) –3 papers AHM04 (2), Phys. Rev. Lett. –6 conference presentations –AHM04, SC2005, Systems Biology, IPEM(2), MC2004

8. UKGSC: Queue Totals For Month Prior To (10:11 AM, 26/11/2004) Notes Certification still dominant query type General contains GGUS related queries from EGEE/GGUS helpdesk (A separate project for this has now been created) LCG savannah generated queries are increasing through this helpdesk Helpdesk http://www.grid-support.ac.uk support@grid-support.ac.uk

9. NGS Core Services Globus, SRB, Oracle, OGSA-DAI, and others

10. NGS Core Services: Globus Globus Toolkit version 2 –GT 2.4.3 from VDT 1.2 Job submission (GRAM) File transfer (GridFTP) Shell (GSI-SSH) Information Services (MDS/GIIS/GRIS) –Information providers from GLUE schema –Use BDII implementation of MDS2 (as does EGEE)

11. NGS Core Services: SRB Storage Resource Broker from SDSC Location transparent access to storage Metadata catalog Replica management Clients on compute nodes Servers on data nodes Issues/to do: –licensing –MCAT replication and failover

12. NGS Core Services: Oracle Oracle 9i database Only on data nodes Populated by users/data providers Infrastructure maintained by NGS database administrators Accessed directly or via OGSA-DAI

13. NGS Services: OGSA-DAI Developed by UK e-Science projects OGSA-DAI and DAIT OGSA-DQP (Distributed Query Processing) Experimental service based on OGSI/GT3 on Manchester data node only –containment: 2 cluster nodes reserved for development and production –will consider WS-I and WSRF flavours when in final release Uses Oracle underneath User-provided Grid Data Services validated on test system, then transferred to production during scheduled maintenance Early users from e-Social Science (ConvertGrid) Established liaisons with OGSA-DAI team

14. NGS Core Services: other Operated by GOSC for NGS and UK e-Science programme In production: Certificate Authority Information Services (MDS/GIIS) MyProxy server Integration tests and database Cluster monitoring LCG-VO In testing: VOMS EDG Resource Broker In development: Accounting –using GGF Usage Record standard for interchange, software from MCS project and TeraGrid Portal

15. Expansion Adding new resource providers

16. Expansion Resource providers join the NGS by Defining level of service commitments through SLDs Adopting NGS acceptable use and security policies Run compatible middleware –as defined by NGS Minimum Software Stack –and verified by compliance test suite Support monitoring and accounting Two levels of membership 1.Affiliation a.k.a. connect to NGS 2.Partnership

17. Affiliation Affiliates commit to: running NGS-compatible middleware –as defined in NGS Minimum Software Stack –this means users of affiliate’s resources can access these using same client tools they use to access NGS a well-defined level of service and problem referral mechanisms –SLD approved by NGS Management Board and published on NGS web-site providing technical, administrative, and security (CERT) contacts providing an account and mapping for daily compliance tests (GITS++) accepting UK e-Science certificates maintaining baseline of logs to assist problem resolution

18. Partnership Partners commit to same as affiliates, plus: making “significant” resources available to NGS users –creation of accounts/mappings –in future, VO support, pool accounts, etc recognise additional CAs with which UK e-Science programme has reciprocal agreements publish/provide additional information to support resource discovery, brokering ability to compile code for computational resources Bristol and Cardiff going through certification process now. supported by “buddies” and NGS-Rollout list useful feedback on viability of NGS Minimum Software Stack

19. Benefits Affiliation: NGS brand –certified NGS-compatible better integrated support for local users who also access NGS facilities assistance/advice in maintaining NGS-compatibility over time Partnership: higher brand quality representation on NGS management board –either direct, or through regional or functional consortia NGS brand must be valuable to make this work.

20. User Support

23. Provide first point of contact support Contact point between other helpdesks –Or provide helpdesk facilities for other sites –Users input queries at a range on places behind the scenes collaboration user gets answer back from where they asked it Develop Support relationship with technical expertise at sites

24. StatusAgents InformedUser InformedComment RequestNew query submitted OpenXXQuery number distributed Stalled XX Rejected XX Agent Replied XX Agent email response automatically informs user and associated agents User Replied X User email response automatically informs associated agents Resolved XQuery answer Closed Can be reopened Possible ticket status and order for Helpdesk Tickets

25. An audit trail of all actions performed on a query can be viewed from the History button on the Details screen. View Contact History on the Edit screen shows all queries associated with a user Query History

26. Inter-Helpdesk Development e.g. GGUS

27. Upgrade server to FootPrints v6.5, includes new user portal interface to access knowledge base without logging in. Make server SSL-enabled i.e. httpsuser portal Explore use for other areas of e-Science eg GridPP, OMII Populate UKGSC knowledge base with additional solutions and FAQs. Publicise web interface to users and include web link in email notifications to users. Note: Users logon to FootPrints with their email address and register a password. Create additional business rules to automate procedure and add reports to satisfy SLAs etc. Review best practice for interactions with other e-Science helpdesk systems, internally and externally (e.g. GGUS relationship) Helpdesk Plans

28. Recent Developments and Summary

29. Recent Developments NGS newsletter OMII helpdesk Resource Broker (LCG) available for early adopters TeraGrid –UK certificates accepted on TeraGrid –Looking at INCA monitoring with Jenny Schopf Trial of accounting software from MCS project –using GGF Usage Record draft standard (as EGEE) Meeting with Open Science Grid’s iGOC Collecting user exemplars –Agreed form of acknowledgement Workshop towards an NGS user induction course. Need: –Easier access (e.g. user interface machines and portals) –T-infrastructure

30. Summary TODAY 4 core nodes operational 150 Users registered (50 since 1 September ’04) Grid enabled – Globus v2 (VDT distribution v1.2) at present BDII information service (GLUE + MDS Schemas) Data Services – SRB, Oracle, OGSA-DAI Growing base of user applications MyProxy and CA services VO Management Software – LCG-VO User support: Helpdesk Next… Other Middleware [gLite/OMII etc…] NGS Portal Resource Broker SRB production service Accounting Continued expansion

31. GOSC and NGS Process decision/testing/transition/deployment of Web Service Grid Middleware

32. Maintaining Compatibility Operating a production grid means valuing robustness and reliability over fashion. NGS cares about: –alignment/compatibility with leading international Grid efforts –special requirements of UK e-Science community –easy migration/upgrade paths –proven robustness/reliability –based on standards or standards-track specifications NGS cannot support everything Everyone wants service-oriented grids –but still settling out: WS-I, WS-I+, OGSI, WSRF, GT3, GT4, gLite Caution over OGSI/WSRF has led to wide convergence on GT2 for production grids and hence some inter-Grid compatibility –but there are potentially divergent forces at work Significant changes to NGS Minimum Software Stack will require approval by NGS Management Board on conservative time scales

33. Strategic Framework GOSC/NGS  UK e-Science project –support other UK (e-)science projects International Compatibility –EGEE European infrastructure (and possible funding) LHC at most UK universities –only user group who want to build the grid –GridPP committed to common w/s plan in 2005 GEANT –Others TeraGrid – US cyberinfrastructure $$$ (unlikely to pay us) Open Science Grid – will develop compatibility with LCG RoW e.g. China –Want use other software, but must be EGEE compatible –Also driven by user requirements –Sets framework for relationship with OMII and others

34. Process for Moving Forward 1.New developments evaluated by ETF –must have some longer term support likely 2.User requests treated on case by case basis 3.NGS Technical Board consider against needs user demand new functionality improved functionality improved security/performace/managability 4.Proposal brought to GOSC Board Prepared by GOSC “executive” N.Geddes, S.Pickles, A.Richards, S.Newhouse

35. “User requests treated on case by case basis” Already see users running web services in user space Exactly what we want … but … –Potential security risks Change Conditions of Use to reflect user responsibilities Require secured web services (X509) Encourage auditing and audit trails –With time limits –Services run “at risk” –Services lead to significant system load run on head node (or other specialised node) –Full support only when “approved”

36. The GOSC Board Director, GOSC (Chair)Neil Geddes Technical Director, GOSCStephen Pickles Collaborating Institutions CCLRCProf. Ken Peach LeedsProf. Peter Dew OxfordProf. Paul Jeffreys ManchesterMr. Terry Hewitt Edinburgh/NeSCProf. Malcolm Atkinson UKERNADr. Bob Day London Collegetbd ETF ChairDr. Stephen Newhouse GridPP Project LeaderProf. Tony Doyle OMII DirectorDr. Alistair Dunlop EGEE UK+I Federation Leader Dr. Robin Middleton HEC LiaisonMr. Hugh Pilcher-Clayton Also invited e-Science User Board Chair. Prof. Jeremy Frey Director, e-Science Core ProgrammeDr. Anne Trefethen

38. Expectations Little demand for GT4 –expect usability much better than GT3 –Watching brief, OGSA-DAI may drive this Indications of Glite improvements in –VOMS, RB, shell, File i/o, data catalog –unlikely to have full ETF assessment by End March –unlikely to all be robust before Q4 OMII job submission –expect good usability, limited functionality –run as user services –problems integrating into Resource Broker and Accounting ? Net result likely to be a vague w/s plan –Hopefully able to focus on some key components