1. Nordic ROC Organization
Gert Svensson, PDC, KTH, Nordic ROC Manager
Nordic ROC and Baltic Grid meeting - Helsinki-
15 June 2009

2. ROC Duties Provide Help Desk facilities (first-level support).
Provide second-level support by helping in the resolution of advanced and specialized operational problems that cannot be solved by site administrators.
If necessary, the ROC will propagate and follow-up problems with higher-level operational or development teams.
Ticket follow-up (ensure that sites work on tickets opened against them).
Respond to tickets from sites in a timely manner.
Manage and support the deployment of gLite middleware on sites.
Registering new sites.
Follow-up on accounting.
Nordic ROC & BG – Helsinki, 15 June 2009

3. Functions and tools Functional Areas Operational tools
Ticketing System

4. Regionalized model What is our target model?
In EGI as much as possible will be regionalized based on NGI:s National Grid Initiatives
ROC:s are responsible for day to day operations, with a minimal organization overseeing them
More efficient
Several NGI:s can have one ROC
Nordic ROC & BG – Helsinki, 15 June 2009

5. Current operational model
Nordic ROC & BG – Helsinki, 15 June 2009

6. Transition r-COD COD Duties to be performed all the time
Duties to be performed periodically
Look at the whole infrastructure
r-COD
Duties to be performed all the time
Only look at sites in own region
Nordic ROC & BG – Helsinki, 15 June 2009

7. Another view
Nordic ROC & BG – Helsinki, 15 June 2009

8. Site responsibilty Adhere to the Operations Procedures Manual
Maintain accurate information in GOCDB
Adhere to the Grid Site Operations Policy
Adhere to the Security and Availability Policy document
Adhere to Service Level Description (SLD)
Deploy supported versions of gLite (or compatible) middleware
Respond to tickets in a timely manner
Nordic ROC & BG – Helsinki, 15 June 2009

9. First line user support - TPM
Provides 1st line support for users together with VO experts
Assigns tickets to appropriate support units
Monitor longstanding open unchanged tickets
Is at the time being a central task
More tickets will be sent directly to ROC in the future
Only cases without natural region will be handled centrally
Follow-up will stay central
Nordic ROC & BG – Helsinki, 15 June 2009

10. First line support function
First-line support in GGUS is called Ticket Process Management (TPM).
The TPM duty is to assign tickets to the right Support Unit (SU).
Assignment must be done in less than one working hour.
TPMs only see 'normal' submitted tickets, i.e. those not assigned automatically (to the ROCs or a few VOs today).
TPMs should recuperate 'forgotten' tickets.
TPMs are notified for action on 2nd and 3rd level of ticket escalation.
TPMs should open savannah entries for middleware problems submitted to GGUS.
Function and models' details in
Antoni | Bosio | Dimou - SA1 F2F Meeting | CERN | 09/06/09

11. User support workflow User Support
Ticket Processing Managers (TPM) analyse the problems reported and assign them to the correct second-level support units.
VOs have support infrastructures to help their users with VO-specific problems. These infrastructures are under their own control. Usually, they are using other tools to support this effort.
The Regional Operations Centres are responsible for dealing with problems arising in their associated resource centres
GGUS benefits from experts spread all over the world for solving issues related to grid security, to networks, and to the interfaces with other grids.

12. User Support Workflow contd.
For VO users and VO specific problems
Mail to
- Solves
- Classifies
- Monitors
Automatic
Ticket Creation
TPM
Grid+VO
experts
VO-specific
Central Application
(GGUS)
VO Support
Units
Middleware
Support Units
Deployment
Operations
Support
ROC
Network
Other Grids
Nordic ROC & BG – Helsinki, 15 June 2009

13. Multi level monitoring framework

14. Terminology SNIC - Swedish National Infrastructure for Computing
Organizing high-performance computing in Sweden
Joint Research Unit in the EGEE III project
NDGF – Nordic Data Grid Facility
an organization for Grids set up by the Nordic Countries
runs the Nordic Tier-1 distributed over 9 sites
develops ARC middleware
most staff distributed in the Nordic countries
Nordic ROC & BG – Helsinki, 15 June 2009

15. NE ROC organisation Two federations: Nordic, Benelux
One distributed ROC
Three sites in Sweden, one in Finland (SNIC + NGDF) and three in Netherlands
GGUS handling and 1:st line support – Regional Operator on Duty (ROD):
Nordic handles Nordic Sites + Baltic Grid
Collaboration between SNIC ROC and NDGF
Netherlands handles the Benelux sites
Rotated among the sites in weekly shifts
TPM duty rotated between all ROC:s
Rotated between sites in the NE ROC
Nordic ROC & BG – Helsinki, 15 June 2009

16. Challenges Distributed ROC Distributed Tier-1
Knowledge of ARC and gLite by different groups
Nordic ROC & BG – Helsinki, 15 June 2009

17. Regular meetings Meetings Nordic ROC meeting Thursday 10.00 Phone
EGEE & WLCG Joint Operations meeting Monday Phone
EGEE SA1 meeting each second Tuesday Phone
NGDF meeting Friday Jabber
Nordic ROC & BG – Helsinki, 15 June 2009

18. Things to discuss How do we improve communication?
Contact site directly, operations directors etc?
Provide more help
SLA – EGEE requires Service Level Agreements
How do we handle that?
75 % availability over each month
What should we do when site doesn’t respond?
Nordic ROC & BG – Helsinki, 15 June 2009