1. GGUS at PEB –- 29.07.03 –- page 1 LCG Klaus-Peter Mickel, GridKa Karlsruhe LCG-PEB-Meeting (29.07.03) The Global Grid User Support Model (Report of GDB Working Group 5 and of related GDB discussions) 1. The history of the WG5 report 2. The Global Grid User Support Model 3.Procedures and Organization 4. Some screenshots from www.ggus.org Klaus-Peter Mickel GridKa Karlsruhe (Forschungszentrum Karlsruhe, Germany)

2. GGUS at PEB –- 29.07.03 –- page 2 LCG Klaus-Peter Mickel, GridKa Karlsruhe The Global Grid User Support Model 1. The history of the WG5 report Grid Deployment Board (GDB): 09.12.02:Formation of WG5, mandate: Recommend the initial LCG-1 user support model 06.02.03:Presentation of first ideas to the GDB 21.02.03:WG5 Report v 1.0 is finished and in the web 10.03.03:GDB: Considerable discussion on the scope of the support model 10.04.03:GDB: GridKa (Karlsruhe) will take responsibility for the user support activity 08.05.03:Some ideas concerning the GGUS web portal presented to the GDB 23.05.03:WG5 Report v 2.0 is finished and in the web 10.06.03:Online presentation of www.ggus.org to the GDB 29.07.03:GGUS model presented to the PEB

3. GGUS at PEB –- 29.07.03 –- page 3 LCG Klaus-Peter Mickel, GridKa Karlsruhe 2. The Global Grid User Support Model (GGUS Model - WG5) Members of the WG5: Luca dell'Agnello (Italy) Ian Bird (Cern) John Gordon (UK) Thomas Kachelhoffer (France) Klaus-Peter Mickel (Germany) Laura Perini (Atlas) People at GridKa (Karlsruhe), who have written the draft: Hans Baer Roland Pietschmann Wolfgang Thoene The WG5 is completely convinced, that effective user services in a grid environment are absolutely essential for a greater success of the Grid.

4. GGUS at PEB –- 29.07.03 –- page 4 LCG Klaus-Peter Mickel, GridKa Karlsruhe WG5 – The Mandate Recommend the initial LCG-1 user support model, including:  Define the scope of responsibilities for a call center and/or helpdesk type facility serving LCG;  Define the call center – one place or many ? – and define the process for communication to ensure problem resolution tracking;  Define acceptable user expectations (or SLA, service level agreements), including the framework within which to implement these;  Define how such a facility interacts with local support services at sites providing grid resources;  Recommend procedures and tools to track and coordinate distributed/grid problem resolution.

5. GGUS at PEB –- 29.07.03 –- page 5 LCG Klaus-Peter Mickel, GridKa Karlsruhe WG5 - GGUS – General Approach  We need a support for 24 hours on all days of the year  The WG5 has proposed three different support centers in three different time zones  The WG5 has proposed one single point of access for all global distributed users:  One web portal (e.g. www.ggus.org)  One mail address (e.g. mail@www.ggus.org)  should be switched to the active support center at a time  One phone number (e.g. +41-22-76-12345)  should be switched to the active support center at a time  Communication between these three support centers should be organized with a ticketing system via XML and with replicated databases

6. GGUS at PEB –- 29.07.03 –- page 6 LCG Klaus-Peter Mickel, GridKa Karlsruhe GGUS – General Approach cont'd But:  Many people say that 90 percent of all reported problems are experiment specific and not Grid related and that the support people at the Global Grid User Support Centers would not be able to handle these experiment specific problems in a right and quick way.  The GDB has proposed, that the first line of support should be at experiment specialists, and that the Global Grid User Support should only be the second line of support.  But nevertheless: We need a Global Grid Support for 24 hours on all days of the year;  And therefore: The WG5 has proposed at least three different central support centers in three different time zones.

7. GGUS at PEB –- 29.07.03 –- page 7 LCG Klaus-Peter Mickel, GridKa Karlsruhe GGUS – General Approach (cont'd) The Grid Support Center on duty as the single point of access for all level 2 problems (since the first level support should be gi- ven by the local experiment specialists) Access via an interface to e.g. Savannah or via the web Three different support centers in three different time zones Europe 8 - 16 h GMT US 16 - 24 h GMT Far East 0 - 8 h GMT Local Grid operations Customer/Experiment Level Support LevelLocal operations Level

8. GGUS at PEB –- 29.07.03 –- page 8 LCG Klaus-Peter Mickel, GridKa Karlsruhe Local Operations Level: At Central Grid Operation Center and at each T-1-C (and also at each T-2-C?) e.g.: Problem solving Maintenance Local services Resource management Preventive activities Problem announcements The Support Model — three levels Customer/Experi- Problem orientedInformation oriented ment Level: Submit a problem Ask for current Grid status, Track a problem documentation, training Support Level: At least three identical support centers with: Helpdesk application User, ticket and resource data base Knowledge base On call service outside the working hours

9. GGUS at PEB –- 29.07.03 –- page 9 LCG Klaus-Peter Mickel, GridKa Karlsruhe 3. GGUS – Procedures 1.An user sends a request to "his" experiment specialists (on every possible way (web, mail, phone))  We need four experiment specialists' teams at each T-1-C ! 2.The request is filed and numbered within the experiment specific bug tracking system (e.g. Savannah). This ticket belongs to the experiment's tracking system. 3.Someone at the experiments support level decides whether it is a Grid problem or not. 4.In case it is a Grid problem, the ticket is assigned to the GGUS team. 5.This assignment should be done using a XML interface or a “formatted” email to a central GGUS account. 6.At least the following data should be interchanged: User / Experiment / Problem description / Actions so far

10. GGUS at PEB –- 29.07.03 –- page 10 LCG Klaus-Peter Mickel, GridKa Karlsruhe GGUS – Procedures (cont'd) 7.For Grid related problems GGUS will create a ticket within its own ticketing application and resend the ticket information to the Savannah system (using a XML message or an “formatted” email) with its own ticket number, and send an email to the user telling him his ticket number and how he can track this ticket. This ticket belongs to the GGUS ticketing system. 8.GGUS will work on the problem and whenever there is an update, an email is sent to the user and the Savannah ticket is updated using XML or “formatted” email 9.When the problem is solved a final email is sent to the user and a final update is sent to Savannah 10.If the user complains, asks or has additional questions or needs further information, he can use the GGUS web portal to look for his ticket

11. GGUS at PEB –- 29.07.03 –- page 11 LCG Klaus-Peter Mickel, GridKa Karlsruhe Organization of the GGUS Centers  There is one overall Grid Portal for all level 2 problems.  Behind it are hidden at least three real support centers with well trained support people for all grid related problems.  The (at least) three real support centers should use the same software and hardware systems.  At all real support centers, there must be the identical (replicated) databases and Grid user directories.  All systems and support people are tripled; therefore a very high reliability is reachable.  As a ticketing system the GridKa people have evaluated and proposed the commercial product Remedy (because it is well known, wide spread and reliable).  One should establish a knowledge base with all solutions of all user problems up to now. With such a knowledge base it should be possible to solve about 50 % of all problems "out of the box".

12. GGUS at PEB –- 29.07.03 –- page 12 LCG Klaus-Peter Mickel, GridKa Karlsruhe The Global Grid User Support Model 4. Some screenshots from www.ggus.org Remarks:-GridKa has reserved this domain for LCG -The Grid portal www.ggus.org is working at present in an early test version -The German HEP users at GridKa are beginning to test this GGUS system; for that we have established eight experiment specialists' mailing lists - one for each GridKa HEP experiment -The authorization to use this portal from everywhere should be controlled later on by the central user administration (maybe a LCG wide LDAP directory structure)

13. GGUS at PEB –- 29.07.03 –- page 13 LCG Klaus-Peter Mickel, GridKa Karlsruhe

14. GGUS at PEB –- 29.07.03 –- page 14 LCG Klaus-Peter Mickel, GridKa Karlsruhe

15. GGUS at PEB –- 29.07.03 –- page 15 LCG Klaus-Peter Mickel, GridKa Karlsruhe

16. GGUS at PEB –- 29.07.03 –- page 16 LCG Klaus-Peter Mickel, GridKa Karlsruhe

17. GGUS at PEB –- 29.07.03 –- page 17 LCG Klaus-Peter Mickel, GridKa Karlsruhe

18. GGUS at PEB –- 29.07.03 –- page 18 LCG Klaus-Peter Mickel, GridKa Karlsruhe mail@ggus.org

19. GGUS at PEB –- 29.07.03 –- page 19 LCG Klaus-Peter Mickel, GridKa Karlsruhe

20. GGUS at PEB –- 29.07.03 –- page 20 LCG Klaus-Peter Mickel, GridKa Karlsruhe

21. GGUS at PEB –- 29.07.03 –- page 21 LCG Klaus-Peter Mickel, GridKa Karlsruhe

22. GGUS at PEB –- 29.07.03 –- page 22 LCG Klaus-Peter Mickel, GridKa Karlsruhe