Grid Middleware Tutorial / Grid Technologies IntroSlide 1 /14 Grid Technologies Intro Ivan Degtyarenko ivan.degtyarenko dog csc dot fi CSC – The Finnish IT Center for Science June 3 rd, 2008 Introduction to Grids and NorduGrid

Grid Middleware Tutorial / Grid Technologies IntroSlide 2 /14 Contents ● Part 1: Introduction to Grids and NorduGrid  What is the Grid?  Role of Middleware  Accessing the Grid  Grid reality  NorduGrid collaboration / M-grid as an example ● Part 2: NorduGrid middleware: ARC  Submitting jobs in NorduGrid, writing job description files  Simple file transfers  Monitoring jobs using the Grid Monitor graphical interface

Grid Middleware Tutorial / Grid Technologies IntroSlide 3 /14 Hierarchy of the IT Resources desktops in-house resources regional computer centers grid infrastructures international HPC centers

Grid Middleware Tutorial / Grid Technologies IntroSlide 4 /14 What is the Grid? no clear definition: Compute Grids, Science Grids, Data Grids, Cluster Grids etc. must be evaluated in terms of the applications, and scientific results that it delivers, not its architecture the key concept is the ability to negotiate resource-sharing arrangements among a set of participating parties and then to use the resulting resource pool a basic definition for grid might look like: "a hardware and software infrastructure that provides dependable, consistent, pervasive and inexpensive access to high-end computational capabilities"

Grid Middleware Tutorial / Grid Technologies IntroSlide 5 /14 What is the Grid? (cont.) Ian Foster, What is the Grid? A three point checklist, Grid Today, 1(6), A Grid is a system that: coordinates resources that are not subject to centralized control: a Grid integrates and coordinates resources and users that live within different control domains using standard, open, general-purpose protocols and interfaces: a Grid is built from multi-purpose protocols and interfaces that address such fundamental issues as authentication, authorization, resource discovery, and resource access deliver nontrivial qualities of service: the utility of the combined system is significantly greater than that of the sum of its parts (response time, throughput, availability, security etc.)

Grid Middleware Tutorial / Grid Technologies IntroSlide 6 /14 Role of Grid Middleware user applications grid middleware Grid resource Grid resource Grid resource Grid resource

Grid Middleware Tutorial / Grid Technologies IntroSlide 7 /14 Grid fundamental services implemented in the middleware resource discovery and monitoring job submission and management brokering and data services information services resource management

Grid Middleware Tutorial / Grid Technologies IntroSlide 8 /14 Computational Grids: resource sharing principle Goal is to minimize waste of resources: empty nodes are always available for use (dynamic sharing)

Grid Middleware Tutorial / Grid Technologies IntroSlide 9 /14 Accessing the Grid: Authentication one needs to be authenticated − every grid user holds a X.509 certificate issued by a Certificate Authority −being authenticated by a site means that the Issuer CA of your credential is trusted by the Grid resource −certificates to be updated annually authentication and authorization are decoupled processes user identification is called authorization

Grid Middleware Tutorial / Grid Technologies IntroSlide 10 /14 ● users are grouped into Virtual Organizations (VO)  based on a common research area, country, funding agency, project, etc.  the same user can belong to several VOs ● resource providers grant access to VOs  effective managing comparing to individual users  implies trust towards the organization managing the VO  for instance: e.g. in the M-grid the users of each site form one VO, and we could combine all to a larger "M-grid VO" when negotiating with external parties Accessing the Grid: Authorization granting resource access is called authorization

Grid Middleware Tutorial / Grid Technologies IntroSlide 11 /14 Grid Security WARNING! When working on the Grid, you must accept that some information on your jobs and on your Grid identity is made public, i.e your name / affiliation IP address of your client computer job names and duration runtime environment others

Grid Middleware Tutorial / Grid Technologies IntroSlide 12 /14 Grid Reality ● a convenient way of sharing resources ● “free” computational power available to those willing to put in the effort to make use of it ● production grids in place and used  Nordugrid, EGEE, DEISA ● different middleware, not normally interoperable ● bureaucracy involved ● the grid won't magically bind your software together

Grid Middleware Tutorial / Grid Technologies IntroSlide 13 /14 NorduGrid Collaboration ● past: Grid pilot project by Nordunet in 2001  implemented a production Grid system working non stop since May 2002 ● present: a community around open source Grid middleware: NorduGrid ARC  national Grids (e.g. M-grid, SweGrid, NorGrid), users also outside the Nordic countries  real users, real applications  funding for further development (KnowARC, NDGF) ● open for anyone to participate

Grid Middleware Tutorial / Grid Technologies IntroSlide 14 /14 The Finnish Material Sciences Grid (M-grid) ● goal: throughput computing capacity mainly for the needs of physics and chemistry researchers  opened to all CSC customers in Nov 2005, not limited by the field of research ● joint project between seven Finnish universities, Helsinki Institute of Physics and CSC  partners mainly laboratories and departments, not university IT centers ● first large initiative to put Grid middleware into production use in Finland  uses NorduGrid ARC middleware ● platform: Linux based PC clusters