1. BiG Grid: the Dutch production grid David Groep, NIKHEF Graphics: Real Time Monitor, Gidon Moont, Imperial College London, see http://gridportal.hep.ph.ic.ac.uk/rtm/ Jan Just Keijser, Nikhef janjust@nikhef.nl 14 October 2010

2. Slide courtesy: GridCafe.org and Bob Jones, EGEE (www.eu-egee.org)

3. “If the customer calls it a grid, it is a grid”

4. Why would we need a grid? Reference: Bible5MByte X-ray image5MByte/image Functional MRI1GByte/day Bio-informatics databases500GByte each Refereed journal papers1TByte/yr Satellite world imagery5TByte/yr US Library of Congress20TByte Internet Archive 1996-2002100TByte Particle Physics 20051PByte/yr Particle Physics Today: LHC20PByte/yr Enhanced Science needs more and more computations and Collected data in science and industry grows exponentially 1 Petabyte = 1 000 000 000 Megabyte

5. From research to e-Science The use of computers and ICT in experiments is still increasing Some experiments would no longer be possible without computers Because ICT is becoming an integral part of the experiment, the experiment itself changes: Research becomes e-Science

6. LHC Computing Large Hadron Collider ‘the worlds largest microscope’ 'looking at the fundamental forces of nature’ 27 km circumference CERN, Genève atom 10 -15 m nucleus quarks ~ 20 PByte of data per year, ~ 60 000 modern PC style computers

8. The ATLAS detector !

10. Grids in e-Science Grid means 'more than one' More than one computer More than one location More than one Field of science More than …

11. More than one organisation National Grid Initiatives & European Grid Initiative At the national level a grid infrastructure is offered to national and international users by the NGIs. BiG Grid is (de facto) the Dutch NGI. The 'European Grid Initiative' coordinates the efforts of the different NGIs and ensures interoperability Circa 40 European NGIs, with links to South America and Taiwan Headquarter of EGI is at the Science Park in Amsterdam

12. Cross-domain and global e-Science grids The communities that make up the grid: not under single hierarchical control, temporarily joining forces to solve a particular problem at hand, bringing to the collaboration a subset of their resources, sharing those at their discretion and each under their own conditions.

13. Grid Infrastructure To bring this about and sustain it requires a persistent infrastructure based on standards Hardware infrastructure clusters, supercomputers, databases, mass storage, visualisation, networks Software infrastructure execution services, workflow, resource information systems, database access, storage management, meta-data Application infrastructure user support, and ICT experts … with domain knowledge Trust and AAA infrastructure authentication, authorization, accounting, billing and settlement

14. Nikhef (NDPF) 2500 processor cores 2000TByte disk 160Gbps network SARA (GINA+LISA) 4800 processor cores 1800TByte disk 2000TByte tape 160Gbps network RUG-CIT (Grid) 120 processor cores 8 800GByte disk 10Gbps network Philips Research Ehv 1600processor cores 100TByte disk 1Gbps network

15. Authentication: Grid certificates Some research institutes are connected to TERENA: Trans- European Research and Education Networking Association TERENA is a 'web of trust' where connected institutes trust each others logins Via the 'TERENA Online Certificate Store' an automated translation is made from the institute username+password to a grid certificate. Grid infrastructure does not make use of usernames and passwords For authentication a Public Key Infrastructure (PKI) with X.509 certificates is used: grid certificates In the Netherlands grid certificates are issued by DutchGrid:

16. Authentication: Grid proxies Sometimes it is necessary to pass on login credentials to a service Users delegate their credentials using a grid proxy A grid proxy (RFC3820) is a “watered-down” version of a grid certificate with a very short lifetime (12 hours, maximum up to 1 week) A grid proxy differs slightly from a regular X.509 certificaat: –Normally only certificates signed by a Certificate Authorities (CA's) are valid. A grid proxy is signed by the user herself. The grid middleware needs to be able to cope with this. –The name (subject) of a grid proxy must be derived from the subject of the users's grid certifcate, e.g. subject : /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser/CN=proxy issuer : /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser identity : /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser type : full legacy globus proxy strength : 1024 bits path : /tmp/x509up_u7651 timeleft : 11:59:40

17. Authentication: Robot certificates For certain services robot certificates can also be used Users receive a username and password for a Grid Portal This portal makes use of a hardware security device for authentication on the grid All portal users are mapped onto a single (robot) grid certificate. Because of this only a limited set of services (applications) may be offered through a portal. Dutchgrid/Nikhef was one of the first organisations to support and offer robot certificates: –Based on Aladdin eToken –Available for Linux, Windows and Mac OS –Both for robot certificates and for user's grid certificates

18. Authorisation: Virtual Organisations To make use of a grid infrastructure a scientist (user) needs to be member of a Virtual Organisation (VO) VOs can exist on both the national and international scale International VOs: – /atlas – /lhcb – /enmr.eu – /ops National VOs: – /lsgrid – /ncf – /vlemed Within a VO multiple roles and groups can exist: – /Role=VO-admin : for VO administration – /Role=SGM : to perform remote Software installations – /lsgrid/Nijmegen – /lsgrid/Wageningen

19. How do I run an application on the (BiG) Grid?

20. NL BE UK Compute Element Workload Management System Grid cluster 1 Grid Information System User delegates credentials & submits job

21. NL BE UK Compute Element Workload Management System Grid cluster Grid Information System 2 3 WMS retrieves information from the Information system and “brokers” the job How do I run an application on the (BiG) Grid?

22. NL BE UK Compute Element Workload Management System Grid cluster Grid Information System 4 5 WMS sends the job to a site CE sends the job to a local cluster

23. How do I run an application on the (BiG) Grid? NL BE UK Compute Element Workload Management System Grid cluster Grid Information System 7 6 8 Job status goes back to CE, WMS User queries the job status and retrieves output

24. Virtual Laboratory for e-Science Avian Alert and FlySafe Willem Bouten et al. UvA Institute for Biodiversity Ecosystem Dynamics, IBED Data integration for genomics, proteomics, etc. analysis Timo Breit et al. Swammerdam Institute of Life Sciences Medical Imaging and fMRI Silvia Olabarriaga et al. AMC and UvA IvI Molecular Cell Biology and 3D Electron Microscopy Bram Koster et al. LUMC Microscopic Imaging group Image sources: VL-e Consortium Partners

25. BiG Grid Image sources: BiG Grid Consortium Partners SCIAMACHY Wim Som de Cerff et al. KNMI LOFAR: LOw Frequency ARray radio telescope MPI Nijmegen: Psycholinguistics

26. Grid especially means scaling up: Distributed computing on many, different computers, Distributed storage of data, Large amounts of data (Giga-, Tera-, Petabytes) Large number of files (millions) This gives rise to “interesting” problems: Remote logins are not always possible on the grid, Debugging a program is a challenge, Regular filesystems tend to choke on millions of files, Storage data is one thing, searching and retrieving turn out to be even bigger challenges. Challenges: scaling up

27. Challenges: security Why is security so important for a Scientific grid? As grid site administrator you are allowing relatively unknown persons to run programs on your computers, All of these computers are connected to the internet using an incredibly fast network, This makes the grid a potentially very dangerous service on the internet

28. What is BiG Grid? Collaborative effort of the NBIC, NCF and Nikhef Aims to set up a grid infrastructure for scientific research. This research infrastructure contains compute clusters, data storage, combined with specific middleware and software to enable research which needs more than just raw computing power or data storage. We aim to assist scientists from all backgrounds in exploring and using the opportunities offered by the Dutch e-science grid. http://www.biggrid.nl

29. http://www.biggrid.nl/ http://www.nikhef.nl/grid/