1. The Grid Production infrastructure
Cristina Vistoli
INFN CNAF

2. INFN-Grid – goals
Promote computational grid technologies research & development: Middleware and grid tools
Through european and national projects
DataGrid, DataTAG, Firb-GRID-it, EGEE, LCG, CoreGRID etc
Internal R&D activities
Deploy – operate - support INFN grid production infrastructure: Grid as “coordinated resource sharing” on a large scale for a multi-institutional and dynamic virtual organisation
Set up the national production grid Infrastructure open to the national research community
FIRB: Grid.it – astrophysic, geophysic, biomedicine, computational chemistry etc
Reserch community and industry

3. INFN-Grid – goals
Provide operation and support of the EGEE/LCG production infrastructure
Promote dissemination activity to ‘gridify’ scientific applications
GILDA testbed
Genius portal

4. INFN-GRID partecipation to EGEE
EGEE SA1 – infrastructure operation and support
ROC – Regional Operation Center
CIC - Core Infrastructure Center
EGEE JRA1 – IT-CZ cluster
Workload management system
Resource access - CE – accounting - policy
VOMS
EGEE NA4/NA3/NA2/NA5
HEP application
Generic application
Dissemination

5. INFN-GRID partecipation to LCG
LHC Computing Grid main sites: T1 and n*T2
LHC Experiments and applications support
Operation and deployment of the Grid infrastructure (national and international)

6. INFN-GRID participation to CoreGRID
The CoreGRID Network of Excellence (NoE) aims at strengthening and advancing scientific and technological excellence in the area of Grid and Peer-to-Peer technologies
Grid Information and Monitoring Services
Knowledge & Data Management

7. INFN-GRID partecipation to GRID.IT/FIRB
Set up the national production grid Infrastructure open to the national research community
Grid management and support tools - system
First tools in production
R&D on
Resource Utilization Policies
Data Management
Scientific Data Base grid integration
Middleware porting

8. Italian – Grid (Site/resource map)
INFN VO
CMS
Atlas
Alice
LHCb
Babar
VIRGO
grid.it resources and VO
TRENTO
MILANO
UDINE
TORINO
PADOVA
LNL
PAVIA
FERRARA
TRIESTE
National
Grid )
GENOVA
PARMA
CNAF
BOLOGNA
PISA
FIRENZE
S.Piero
PERUGIA
LNGS
ROMA
ROMA2
L’AQUILA
LNF
SASSARI
NAPOLI
BARI
SALERNO
LECCE
CAGLIARI
COSENZA
PALERMO
CATANIA
LNS

9. Grid-it Status 22 Resource Centres
1 Tier1 : CNAF
4 Tier2: Roma1(2), Milano, Torino, LNL
14 siti INFN: Bologna(2), Bari, Catania, Ferrara, Firenze,Lecce, LNF, Napoli (3), Padova, Perugia, Pisa, Pavia, Roma2, Trieste, Cagliari
3 siti non INFN: INAF-TS, Uni-Na, Sns-Pisa
Servizi : RBs, BDIIs, VOMS, VO-LDAP, Gridice servers, RLS……

10. INFN-GRID: Resources and supported VOs
(**) Hyperthreaded

11. INFN-GRID Release INFN-GRID is a customized release of LCG
All resources are fully managed via LCFGng;
INFN-GRID does not support the middleware installation without LCFGng;
Change with the next release based on SL3:YAIM and Quattor
INFN-GRID release is based upon the official LCG and it is 100% compatible;

12. Grid.IT Production Grid: deployment portal
User documentation
site managers documentation
Software repository
Monitoring
Trouble tickets system
Knowledge base

13. INFN-GRID Release Main differences from LCG 2.3.0 to INFN-GRID 2.3.0:
Added support for DAG jobs;
Added support for AFS on the WorkerNodes;
Added support for MPI jobs via home syncronisation with ssh;
Documented installation of WNs on a private network;
Added full function VOMS support:
INFNGRID, CDF, COMPCHEM, PLANCK are completely managed via VOMS server.

14. grid-it … Cnaf/T1, LNL, To, Roma1,Milano, Padova, Napoli,….
Experiment Support
EGEE/LCG CICs
Controllo dei Servizi e dei Resource Centers, procedure di deployment, Produzione Release e certificazione
Grid-it management
Supporto Esperimenti, Virtual Organizations, Applicazioni Scientifiche
CIC-On-Duty
Cnaf/T1, LNL, To, Roma1,Milano, Padova, Napoli,….
Servizi GRID di Esperimento e/o
di infrastruttura: RBs, VOMS, RLS, GIS, Monitoring….
grid-it
Italian Roc
Grid-it Operation-Support …
CERN
Spanish-Grid
UK-Grid

15. Manage the Problem List
Support workflow
FAQ
GOC Tools
GridICE
Gppmon
Site CERT
Gstat
Etc…
Problem 1001
Problem 1002
Problem 1003
Problem 1004
Problem 1005
Problem 1006
Problem 1007
Problem List
Problem 1001
Problem 1002
Problem 1003
Problem 1004
Problem 1005
Problem 1006
Problem 1007
Problem List
Problem 1001
Problem 1002
Problem 1003
Problem 1004
Problem 1005
Problem 1006
Problem 1007
Problem List
Manage the Problem List
DOC
ROC
ROC
ROC
ROC
ROC
RC (site)
RC (site)
RC (site)
RC (site)
RC (site)
RC (site)
RC (site)
RC (site)
RC (site)

16. CIC-On-Duty (P.Veronesi, A.Cavalli)
Shift settimanale di controllo infrastruttura europea
Interazione con Italian ROC e altri ROC europei

17. Riunioni periodiche di persona
Iniziate a fine giugno phone conference periodiche (bisettimanali) di grid di produzione, EGEE-SA1 + site manager
Riunioni periodiche di persona
Realizzazione release di middleware INFN-GRID – Release Team
Gestione strumenti di installazione automatica
Repository software e configurazioni
Integrazione nuove funzionalità e certificazione
Procedure di installazione, guide d’uso etc. sia automatiche che manuali, anche per SL

18. Supporto
Realizzata la ‘checklist del turnista diligente’ con la collaborazione di tutti le sedi della Grid di produzione
Istituiti turni 8.30 – e – 19.30, 5 giorni la settimanadi controllo Grid di produzione, risposta ai ticket, controllo dei problemi riscontrati a livello di CIC, stato dei servizi
2 persone per turno
Report di fine turno per logging delle attivita’ pendenti, chat channel per colloquio durante il turno
Siamo alla 2 settimana di turno, sistemato procedure e srtumenti
siamo pronti per supportare VO
CIC-on duty : turno settimanale, gli output verso l’italia sono gestiti dai turni nazionali

19. Ticketing system INFN-GRID ticketing system is used:
from users to ask questions or to communicate troubles;
from system manager to communicate about common grid tasks (ex: upgrading to a new grid release)
from CMT to system manager to notify a problem
Support Groups are “helper” groups and they exist to resolve the obvious problems arising with the grow of the grid:
Support Grid Services (RB, RLS, VOMS, GridICE, etc) Group;
Support VO Services Group (each for every VO);
Support VOApplications Group (each for every VO);
Support Site Group (each for every site)
Operative Groups Operative Central Management Team (CMT);
Operative Release & Deployment Team;
Users -> Create a ticket
Supporters/Operatives -> Open the ticket
Users and/or Supporters/Operatives -> Update an open ticket
Supporters/Operatives -> Close the ticket

20. EGEE/LCG: Production Grid services
RB-BDII scope all european resources EGEE/LCG
RB/UI with DAG
Service Resources are open to all VOs supported by INFN-GRID and EGEE/LCG
RB: egee-rb-01.cnaf.infn.it
support BIOMED VO

21. Grid-it: Production Grid service
Service Resources are open to all VOs supported
RB-BDII scope Italian Grid
NEW! Resource Broker/UI DAG
prod-rb-01.pd.infn.it

22. Certification activity – TEST ZONE
The Central Management Team is responsible of the resource centers certification: checking the functionalities of a site before joining the site to the production grid.
Although all certification jobs are VO independent, the INFNGRID VO is used to perform these jobs;
In particular are checked:
GIIS' information consistence;
Local jobs submission (LRMS);
Grid submission with Globus (globus-job-run);
Grid submission with the ResorceBroker;
ReplicaManager functionalities;
MPI functionalities
In order to certificate a site the CMT uses dedicated grid services:
RB & BDII: gridit-cert-rb.cnaf.infn.it
In this way we avoid to have an uncertified site in the production grid services;

23. Attivita’ in corso
Sistema di supporto: integrazione in EGEE e copertura supporto distribuito
Evoluzione di Gridice per job monitoring, application monitoring, SLA monitoring, urgente configurazione notifiche
Integrazione di DGAS in INFN-GRID  amministrazione sistema di accounting
Porting di INFN-GRID a SL : nuovo sistema di installazione e configurazione
Operation support infrastruttura EGEE/LCG a ‘rotazione’ tra IT/CERN/UK/FR
Training: corso base e avanzato
Allargamento infrastruttura a sedi non INFN: Spaci, Enea, etc
Amministrazione Policy
Pre-production service per definire il programma di migrazione a Glite
Middleware certification testbed
Operational requirements per il middleware

24. Useful links INFN Grid INFN production GRID infrastructure
INFN production GRID infrastructure
INFN GRID development projects portal
INFN GridICE
INFN Support
Contact
(management board)
(technical board)
(production grid management team)