1. INFN Grid Project Targets, organisation, procedures
Alfredo Pagano
INFN – CNAF
on behalf of CMT and Italian ROC
Scuola Grid - Martina Franca, Monday 05 November

2. Scuola Grid - Martina Franca, Monday 05 November 2007 - 2
Outline
The EGEE project:
Motivations
Organisation
Infrastructure
The INFN Grid project
How to join:
Registration
Certification
MOU (Memorandum of Understanding)
Scuola Grid - Martina Franca, Monday 05 November

3. Scuola Grid - Martina Franca, Monday 05 November 2007 - 3
The EGEE project
EGEE
1 April 2004 – 31 March 2006
71 partners in 27 countries,
federated in regional Grids
EGEE-II
1 April 2006 – 31 March 2008
91 partners in 32 countries
Objectives
Large-scale, production-quality grid infrastructure for e-Science
Attracting new resources and users from industry as well as science
Maintain and further improve “gLite” Grid middleware
Scuola Grid - Martina Franca, Monday 05 November

4. Scuola Grid - Martina Franca, Monday 05 November 2007 - 4
EGEE Activities
Service Activities
SA1 – Grid Operations, Support and Management (CERN)
SA2 – Networking Support (CNRS)
SA3 – Integration, Testing and Certification (CERN)
Joint Research Activities
JRA1 – Middleware Re-engineering (INFN)
JRA2 – Quality Assurance (CS-SI)
Networking Activities
NA1 – Management (CERN)
NA2 – Dissemination, Outreach and Communication (CERN)
NA3 – Training and Induction (UEdin)
NA4 – Application Identification and Support (CNRS)
NA5 – Policy and International Cooperation (GRNET)
Scuola Grid - Martina Franca, Monday 05 November

5. Scuola Grid - Martina Franca, Monday 05 November 2007 - 5
EGEE Applications
Multitude of applications from a growing number of domains
Astrophysics
Computational Chemistry
Earth Sciences
Financial Simulation
Fusion
Geophysics
High Energy Physics
Life Sciences
Multimedia
Material Sciences …
Scuola Grid - Martina Franca, Monday 05 November

6. Scuola Grid - Martina Franca, Monday 05 November 2007 - 6
EGEE operations
- Operations Coord. Centre (OCC)
- Regional Operations Centres (ROC)
Front-line support for user and operations issues
Provide local knowledge and adaptations
One in each region – many distributed
Manage daily grid operations – oversight, troubleshooting, “Operator on Duty”
Run infrastructure services
Scuola Grid - Martina Franca, Monday 05 November

7. EGEE Production Resources
EGEE today:
> 180 sites, 39 countries
> 30,000 processors, > 9 PB storage
> 30K jobs/day
Scuola Grid - Martina Franca, Monday 05 November

8. Scuola Grid - Martina Franca, Monday 05 November 2007 - 8
EGEE Production Usage
~17.5 million jobs run (6450 cpu-years) in 2006;
Workloads of the “not HEP VOs” is now significant – approaching 8-10K jobs per day; and 1000 cpu-months/month
one year ago this was the overall scale of work for all VOs
Scuola Grid - Martina Franca, Monday 05 November

9. EGEE Related projects & other Grids
Potential for linking ~80 countries
Scuola Grid - Martina Franca, Monday 05 November

10. Scuola Grid - Martina Franca, Monday 05 November 2007 - 10
EU Related projects
Name
Description
BalticGrid
EGEE extension to Estonia, Latvia, Lithuania
EELA
EGEE extension to Brazil, Chile, Cuba, Mexico, Argentina
EUChinaGRID
EGEE extension to China
EUMedGRID
EGEE extension to Malta, Algeria, Morocco, Egypt, Syria, Tunisia, Turkey
EU-IndiaGrid
EGEE extension to India
eIRGSP
Policies
ETICS
Repository, Testing
OMII-Europe
to provide key software components for building e- infrastructures;
BELIEF
Digital Library of Grid documentation, organisation of workshops, conferences
BIOINFOGRID
Biomedical
Health-e-Child
Biomedical – Integration of heterogeneous biomedical information for improved healthcare
ICEAGE
International Collaboration to Extend and Advance Grid Education
Scuola Grid - Martina Franca, Monday 05 November

11. INFN Grid: A brief Introduction
INFN Grid activities:
Special project INFNGrid [since 2000]
INFN member of:
DataGrid (EDG), funded by UE [ ]
DataTAG, funded by UE [ ]
LHC Computing Grid (LCG), CERN [since 2002]
EGEE, funded by UE [ ]
EGEE-II, funded by UE [ ]
Grid.IT project, funded by MIUR/FIRB [ ]
Production Quality
Grid
Testbeds
Scuola Grid - Martina Franca, Monday 05 November

12. Scuola Grid - Martina Franca, Monday 05 November 2007 - 12
The INFN-GRID Project
February 2000: The Board of Directors approve the INFN Grid project
Large size : 20 Italian Sites, ~100 people, ~ 50 FTE’s
Collaboration between physicists, sw engineers, computer professionals and computer scientists (CS Dep. of Universities of VE, PD, BO, CT, TO,…), CNR, and Industries
Datamat SPA and Nice major contributors of the joint developments
Focused on the preparation of the INFN LHC comp. infrastructure
…but with the goal of developing a new set of “standard” services and protocols to allow resource sharing in different administrative domains as the server Http and HTML are at basis of information sharing
taking into account the requirements of other sciences
Biology (PD) and Earth Observation (Esrin-ESA-Frascati)
INFN Grid has been and is the national container for INFN to coordinate the contribution to all EU and International Grid projects and to the GGF standardization
Early R&D in Italy include work done in ISUFI (University of Lecce)
->see S-PACI
INFN Grid consider to have accomplish his main goal: All LHC experiments and many other VOs rely on grids for their daily work
Scuola Grid - Martina Franca, Monday 05 November

13. Scuola Grid - Martina Franca, Monday 05 November 2007 - 13
The Italian Production Grid:
a sum of grids
~5000 CPUs
950TB (Disk ) TB (Tape)
40 ‘resource centers’:
INFN Grid + SPACI + ENEA + 5 RCs:
Istituto Tecnologie Biomediche – CNR/BARI (LIBI Project)
PERUGIA University
Istituto Linguistica Computazionale CNR-PISA
Scuola Normale Superiore – PISA
ESA-ESRIN
Significant expansion foreseen thanks to:
Recent PONs TriGrid, PI2S2 Cybersar, Scope, Cresco
Scuola Grid - Martina Franca, Monday 05 November

14. Southern Partnership for Advanced Computational Infrastructure
SPACI
1.5 Tflops
ISUFI/CACT
Center for Advanced Computing
Technologies
University of Salento
Director: Prof. Giovanni Aloisio
IA64 (Itanium 2)
DMA/ICAR
Dept. of Mathematics and Applications
University of Naples “Federico II” & ICAR
(Section of Naples)
Director: Prof. Almerico Murli
MIUR/HPCC
Center of Excellence for
High Perfomance Computing
University of Calabria
Director: Prof. Lucio Grandinetti
Scuola Grid - Martina Franca, Monday 05 November

15. Scuola Grid - Martina Franca, Monday 05 November 2007 - 15
GEANT
CNR
Tor Vergata
Access to not standard platform IAX – IRIS
(afs pool account, lcmaps, yaim customized)
Scuola Grid - Martina Franca, Monday 05 November

16. Managed Core SERVICES Grid.it Production Grid: on
SCIENTIFIC LINUX 3  SL4
Monitoring
Catalogs
Gridice
LFC
DGAS
GPBOX
MyProxy
VOMS
Accounting
Grid policies
GIS RB
Authorization
Security
Information System
Resource Broker
Scuola Grid - Martina Franca, Monday 05 November

17. Scuola Grid - Martina Franca, Monday 05 November 2007 - 17
GRID Services
Allow you to use the grid resources:
Resource Broker (RB) / Workload Management System (WMS): they are responsible for the acceptance of submitted jobs and for sending those jobs to the appropriate resources
Information System (IS): provides information about the grid resources and their status
Virtual Organization Management System (VOMS): database for the authentication and authorization of the users
Gridice: monitoring of resources, services and jobs
Home Location Register (HLR): database for accounting data (usage of resources)
LCG file catalog (LFC): file catalog
File Transfer Service (FTS): file transfer in an efficient and reliable way
R-GMA: Relational Grid Monitoring Architecture
MonBox: collector for local data of R-GMA
Scuola Grid - Martina Franca, Monday 05 November

18. Italian Grid Services and supported VOs
LHC VOs: atlas alice lhcb cms
Italian VOs: argo, bio, compassit, compchem, cdf, egrid, enea, gridit, inaf, ingv, libi, pamela, planck, theophys, virgo
EGEE VOs: babar, biomed, cdf, esr, geant4, gear, geclipse, magic, zeus
Test VOs: dteam, infngrid, ops
Others: euchina, eumed, euindia, cyclops
10 RB: 3 Italian Scope, 3 EGEE Scope, 1 CMS Scope, 1 Euchina Scope, 1 eumed Scope, 1 SFT ADmin Scope
4 WMSLB: 2 EGEE Scope, 1 cms scope, 1 ATLAS scope
6 BDII
2 VOMS servers
1 LFC server
5 gridice servers
12 HLR
New services with release INFNGRID 3.1
WMS, LB, WMSLB dedicated to
atlas, cdf, cms, lhcb
Scuola Grid - Martina Franca, Monday 05 November

19. Scuola Grid - Martina Franca, Monday 05 November 2007 - 19
Regional VOs
VO User
argo
bio
compchem
cyclops
enea
eumed
euchina
Euindia
gridit
inaf
infngrid
ingv
libi
pamela
planck 20
theophys
virgo
Cdf
Egrid
Newest VO: compassit
MAY 2007 TOP USERS:
CDF (~50k proxies) COMPCHEM (205 proxies) PAMELA (464 proxies)
EUCHINA (379 proxies) INFNGRID (Test purposes ~ 25k proxies)
Scuola Grid - Martina Franca, Monday 05 November

20. Scuola Grid - Martina Franca, Monday 05 November 2007 - 20

21. Scuola Grid - Martina Franca, Monday 05 November 2007 - 21

22. s For details… see the presentation:
“ Il Sistema di Supporto INFNGrid ”
on Thursday
Scuola Grid - Martina Franca, Monday 05 November

23. The Central Management Team (CMT)
Release distribution and site certification in Italy
Certification: manual check of functionality and configuration of site services before including the site into the Italian production grid. For example:
Information System data consistence
Local jobs submission (LRMS)
Grid submission with Globus (globus-job-run)
Grid submission with the EGEE Resource Broker
ReplicaManager functionalities 
Certification is based on dedicated grid services located at CNAF
Scuola Grid - Martina Franca, Monday 05 November

24. Central Management Team (CMT) Shifts
About 20 supporters perform a checking activity composed of 2 shifts per day,from Monday to Friday, with 2 persons per shift, during which a report is compiled:
Checking of the grid status and problem warning, tailing them until their solution if possible
Doing site certification during the deployment phases
Check of the open tickets and pressure on the expert/site- managers for a quick resolution of problems
Scuola Grid - Martina Franca, Monday 05 November

25. Scuola Grid - Martina Franca, Monday 05 November 2007 - 25
User and site support
EGEE make use of the GGUS
(Global Grid UserSupport) ticketing system
Each ROC utilizes different tools interfaced
to GGUS in a bidirectional way.
By means of Web services, it is possible to:
Transfer tickets from the global to regional
system
Transfer tickets from the regional to
the global system
The user groups, whose tickets will be addressed, are defined
in both GGUS and the regional systems
Italian Roc ticketing system: XOOPS/xHelp
Scuola Grid - Martina Franca, Monday 05 November

26. Scuola Grid - Martina Franca, Monday 05 November 2007 - 26
Access to the GRID
Access by means of an User Interface (UI). It could be:
A dedicated PC, installed in a similar way to the others grid elements
UI Plug-and-Play (UI PnP), a software you can install on any pc without root privilegies
A web portal:
To access the GRID you need a personal certificate released by a Certification Authority trusted by EGEE/LCG infrastructure: the user authentication is performed through X-509 certificates
To be authorized to submit jobs you have to belong to a Virtual Organisation (VO). A VO is a kind of users group usually working on the same project and using the same application software on the grid.
Scuola Grid - Martina Franca, Monday 05 November

27. Registration of a new site: howto
MOU (Memorandum Of Understanding) via fax 
Site must be inserted into GOCDB, we need
Site name * ex. INFN-CNAF-LHCB
Official name ex.INFN-CNAF-LHCB
Domain name(autocomplete) *
Home URL
Contact address
Contact telephone
Emergency telephone
CSIRT address (CSIRT is the security contact person)
CSIRT telephone
Scuola Grid - Martina Franca, Monday 05 November

28. HOWTO Register a new site
The sitemanager is responsible of:
keeping all the information for his site updated
Also inserting downtime into the GOCDB
subscribing the INFN GRID ticketing system
joining the IRC channel for CMT
installing the INFN Grid Release
registration in the test VOs infngrid and dteam
opening a “ticket” in order to trigger the certification process by the CMT
Scuola Grid - Martina Franca, Monday 05 November

29. Memorandum of Understanding
Provide computing and storage resources. Farm dimensions (at least 10 cpu) and storage capacity will be agreed with each site
Guarantee sufficient man power to manage the site: at least 2 persons
Manage efficently the site resources: middleware installation and upgrade, patch application, configuration changes as requested by CMT and do that by the maximum time stated for the several operation
Answer to the ticket by 24 hours (T2) or 48 hours (other sites) from Mon to Fry
Check from time to time own status
Guarantee continuity to site management and support, also in holidays period
Partecipate to SA1/Production-Grid phone conferences an meetings and compile weekly pre report
Keep updated the information on the GOC DB
Enable test VOs (ops, dteam and infngrid), with a higher priority than other VOs
Eventual non-fulfilment noticed by ROC will be referred to the biweekly INFNGRID phone conferences, then to COLG, eventually to EB.
Scuola Grid - Martina Franca, Monday 05 November

30. CIC Portal: report and broadcast
CIC Portal has been created as a part of the SA1 activity.
It is dedicated to ensure:
to be a management and operations tool
to be an entry point for all Egee actors for their operational needs
to manage the available informations about EGEE VOs and related VOs
to monitor and ensure grid day-to-day operations on grid resources and services
Take care of weekly report that will be sent every Friday by CIC portal.
Till Monday you must fill this report giving a reason for every problem/down/outage your site eventually accused.
Scuola Grid - Martina Franca, Monday 05 November

31. CIC Portal: weekly report
Report settimanale
Scuola Grid - Martina Franca, Monday 05 November

32. Freedom of Choice for Resources
The Freedom of Choice for Resources is a VO Policy enforcement tool, to manipulate top-level BDIIs. It is fully integrated with the SAM ( Service Availiblity Monitoring ) framework. FCR allows the VOs to define a preference on Grid resources, optionally taking the SAM test results in account as well. Only VO responsibles (VO Software Managers, etc.) can get access to the FCR Admin Pages , where they can modify their VO's FCR profile.
They can select the:
set of Critical Tests for all services
set of Site Resources (CEs and SEs) to be used by the VO
set of Central Service Nodes ( Note : this will be used in the future)
Changes are written to the database (that's shared with SAM ), and an LDAP ldif file is created, which the top-level BDIIs download in every 2 minutes in order to apply Site Resources changes.
Scuola Grid - Martina Franca, Monday 05 November

33. Scuola Grid - Martina Franca, Monday 05 November 2007 - 33
The gLite Release
HUGE collection of packages ~ O(1000)
Middleware (services, client tools,)
External dependencies (libraries, ...)
The packages are mainly developed by EGEE JRA1 but also by other projects (VDT, Globus, ...)
Organized in profiles (profile = “node type” = “grid element”)
Each profile is (normally) installed into one node and is composed by one or more services:
Examples of profiles: ComputingElement (CE), Workload Management System (WMS)
Examples of services running on a WMS: glite- networkserver, glite-logging-and-bookeeping, etc...
Each service has one or more daemons
Scuola Grid - Martina Franca, Monday 05 November

34. INFN-GRID Release - why?
The production infrastructure is used by other projects/experiments
Babar, Virgo, CDF, ARGO, Zeus, ...
Additional configuration for the middleware is defind once, at ROC level, to reduce misconfiguration risks:
More VOs: VO servers, poolaccounts, add VOMS certificates, ...
MPI (requested by non-HEP sciences), additional GridICE config (monitor Wns), AFS read-only (CDF requirement), ...
Deploy additional middleware in a non-intrusive way:
Since Nov VOMS, now in gLite; DGAS (DataGrid Accounting System); NetworkMonitor (monitor network connection metrics)
Of course 100% compatibility is mandatory
Scuola Grid - Martina Franca, Monday 05 November

35. INFN-GRID customizations
Additional VOs (~20)
GridICE on WN
Preconfigured support for MPI
WN without home shared but with ssh hostbased authentication
DGAS: accounting
New profile (HLR server) + additional packages on CE and WN
NME (Network Monitor Element)
Quattor (collaboration with CNAF-T1)
NTP
AFS (read-only) on WN (needed by CDF VO)
...
Scuola Grid - Martina Franca, Monday 05 November

36. Scuola Grid - Martina Franca, Monday 05 November 2007 - 36
Short History
LCG
EGEE
EGEE II
LCG 1.0
LCG 2.0
gLite 3.0
2003
2004
2005
2006
2007
2008
1.0
2.0
3.0
INFN-GRID
Scuola Grid - Martina Franca, Monday 05 November

37. Scuola Grid - Martina Franca, Monday 05 November 2007 - 37
Useful links…
Italian grid project:
Italian production grid:
GridOPS:
SAM:
CIC Portal:
GSTAT:
GridICE:
Scuola Grid - Martina Franca, Monday 05 November

38. Scuola Grid - Martina Franca, Monday 05 November 2007 - 38
Credits
Thanks to:
Valeria Ardizzone INFN-CATANIA
Mirco Mazzucato, Paolo Veronesi, Alessandro Paolini, Alessandro Cavalli INFN-CNAF
Cristina Aiftimiei INFN-PADOVA
Scuola Grid - Martina Franca, Monday 05 November

39. Scuola Grid - Martina Franca, Monday 05 November 2007 - 39
The END…
Scuola Grid - Martina Franca, Monday 05 November