1. Introduction to Grids David Boyd Information Technology Department Rutherford Appleton Laboratory D.R.S.Boyd@rl.ac.uk

2. The Grid - what is it? “The grid is an emerging infrastructure that will fundamentally change the way we think about - and use - computing. The grid will connect multiple regional and national computational grids to create a universal source of computing power.” (c.f. electric power grid) The Grid: Blueprint for a New Computing Infrastructure, Foster and Kesselman, Morgan Kaufmann Publishers

3. Promise of ubiquitous computing? “Global virtual computer” Grid “middleware” as the OS of this computer Accessible via a “plug on the wall” Dependable - predictable, sustained performance Consistent - standard interfaces, services and operating parameters Pervasive - widely available via controlled access Inexpensive - perceived cost relative to alternative solutions We will eventually take the Grid for granted

4. Where the Grid is coming from... Work over the last decade on several distinct topics –Metacomputing - combining distributed heterogeneous computing resources on a single problem –Data Archiving - building collections of data on specific topics with open metadata catalogues and well-documented data formats –Collaborative Working - network-based facilities for distributed, concurrent working on shared information –Data visualization - large-scale immersive facilities for 3D visual interaction with data coupled to computational analysis –Network-based information systems - WAIS, Gopher, WWW –Instrumentation control - remote control of experimental equipment and real time data gathering systems

5. The Grid (potentially) offers... A consistent way of combining many types of “computing” resource located anywhere to solve complex problems These resources can be: –networks (LAN, SJ4, Geant(EU), Internet2(US),... ) –computing systems (HPC, MPP, SMP, clusters, workstations, PCs,..) –data storage facilities (tape robots, disk farms, local caches,... ) –visualization / VR environments (VR Centres, CAVEs,..., desktop) –scientific instruments (telescopes, microscopes, synchrotrons, satellites,... ) –.... basically any device which can communicate over the internet

6. But the Grid is not yet... mature, reliable technology a well-understood system a generic solution easy to use capable of sustained operation economically validated... however... I might have said the same about the Web ~7 years ago –and it has since transformed the world’s access to information and created a business revolution... imagine the potential of the Grid over the next 5-7 years

7. The Web vs The Grid The Web supports wide area data/information location and retrieval The Grid supports complete process initiation and execution including any necessary data location and retrieval –offers the potential to carry out significantly large tasks –opens up new capabilities for knowledge generation Currently Grid tools provide a relatively low level of operational control –higher level tools will be developed to automate low level processes –agent technology will eventually support real time dynamic process optimisation

8. Process optimisation By looking at whole processes it will be possible to optimise them using intelligent management strategies –identifying appropriate (by user-specified criteria) computing resources for the task –locating appropriate sources of the relevant data –extracting relevant subsets of this data –assessing options for migrating the data or the task –evaluating the available network service quality –making a decision on a strategy to carry out the user’s task –adapting this to changing circumstances in real time

9. Grid architecture Grid Fabric Applications ArchivesNetworks InstrumentationControl interfacesComputers Grid Services Directory service Resource management Metadata Data access Fault detection Authentication Display devices High-energy physics data analysis Climate modelling Collaborative engineering Parameter studies On-line experiments Application Toolkits High throughput Data intensive computing Collaborative working Remote visualization Remote control Protocols

10. Grid tools and toolkits There are many useful tools and toolkits now available –e.g. Globus, Condor, Legion, SRB, LDAP, OOFS,... Globus provides resource allocation and management (GRAM), information access (MDS), authentication (GSI) Condor provides high throughput computing on distributed networks of workstations with checkpointing Legion is an object-based large scale distributed computing environment designed to handle trillions of objects Storage Resource Broker provides facilities for managing a distributed data repository which includes multiple copies

11. An application of the Grid tomographic reconstruction real-time collection wide-area dissemination desktop & VR clients with shared controls Advanced Photon Source Online Instruments archival storage The Globus project

12. Science-driven Grid applications Environmental science –coupled atmosphere and ocean simulations with long simulated timescales at high resolution Biological science –multiple protein folding simulations to generate statistically valid models of complex molecules Astronomy - “Virtual Observatory” –searching across many instrument-specific data archives to study a new class of object at all wavelengths Materials science –combining and analysing data from different experimental facilities to derive the structure of complex new materials

13. Large Hadron Collider Data Grid Department    Desktop CERN – Tier 0 Tier 1 FNAL RAL IN2P3 622 Mbps 2.5 Gbps 622 Mbps 155 mbps Tier2 Lab a Uni b Lab c Uni n 5 Tier model

14. Some US Grid initiatives Grid Physics Network (GriPhyN) a consortium of universities, was recently given $11.8m by NSF over 5 years for projects in particle physics and astronomy Particle Physics Data Grid is a consortium of major US government laboratories collaborating on Grid developments for the US particle physics programme NASA Information Power Grid is being built to support coupled multi-disciplinary simulations and to provide a national resource for rapid response crisis and disaster management Grid Forum is an open forum of the major Grid development teams with Working Groups on high priority problem areas and a remit to develop standards

15. EU DataGrid project Will link national Grid projects to create a European Grid infrastructure Strong focus on developing middleware tools Application-driven from 3 areas –Particle Physics –Earth Observation –Biosciences 21 partners! (6 main, 15 associated) Close collaboration with US initiatives Likely to lead to several follow-on projects to exploit this infrastructure for social and commercial applications

16. CLRC e-Science programme Project-driven programme to exploit Grid technologies for the benefit of the CLRC science programme and its users by enhancing the CLRC infrastructure and developing Grid expertise –Data Portal - developing a metadata-driven access mechanism for a wide range of scientific data from CLRC facilities and programmes –ATLAS Datastore- Grid-enabling and enhancing the Datastore to Petabyte capacity –Gbit LAN - building a Gbit internal network in preparation for Grid-based applications and imminent SJ4 connection Grid reference system - developing and supporting a reference Grid platform as a basis for internal Grid application projects –plus several science application pilot projects

17. UK Grid programme Proposal to Treasury for £90m as part of SR2000 bid Awaiting news from OST Use science as the “storm troopers” to build a national Grid infrastructure Achieve world-beating science as a result Give UK business a head start on its competitors by early exposure to the technology Transfer the expertise gained into the UK commercial software industry to develop a global market

18. The End