1. 1 AIP Workshop “Hands-On Globus!” 8. 11.2006 Agenda I. What is a grid? II. Globus structure III. Use cases & Hands on! IV. AstroGrid @ AIP: Status and Plans Hands-On-Globus Overview

2. 2 AIP Workshop “Hands-On Globus!” 8. 11.2006 Introduction I: What is a Grid?

3. 3 AIP Workshop “Hands-On Globus!” 8. 11.2006 [cool animation] ‚You have requested historical file number fourty-two dash zero: The Grid. The florishing of human social systems and civilisations has always been accompanied by the development of infrastructure networks for the common good. At the dawn of the second millenium a name was needed to designate a system that used digital networks to share computer resources. Such systems were henceforth called „grids“.‘

4. Altruism, Civilisation, Networks.

5. 5 AIP Workshop “Hands-On Globus!” 8. 11.2006 Modern Communication Networks = information distribution and sharing

6. 6 AIP Workshop “Hands-On Globus!” 8. 11.2006 Old and new networks

7. 7 AIP Workshop “Hands-On Globus!” 8. 11.2006 Next step: Resource sharing: information processing power out of the socket $ submit_job submitting job… requisitioning 7.3 Exaflops… job successful. $ publish_paper compiling & publishing… review process… 2006.nature.447.pp1103-17 OK.

8. 8 AIP Workshop “Hands-On Globus!” 8. 11.2006 What Resources can be shared? Efficient use of: CPU-Power storage space special hardware (sensors, telescopes, robots) installed applications and licenses Advantages: load balancing, scaling redundancy  increased reliability uniform uniform protocol interface for arbitrary resources

9. 9 AIP Workshop “Hands-On Globus!” 8. 11.2006 Again: What is a Grid? A buzz word? A grid distributes, manages and coordiates resource sharing and collaboration without centralised control. A grid uses standardised, open, general-purpose protocols and interfaces. It can be (very) inhomogenious It includes security mechanisms, even though being spread out over multiple diverse „domains“. It is used for professional work It is more than the sum of ist parts and non-trivial. plus  A grid is computers plus the internet plus some more things a.k.a.“meta computing“, „distributed [parallel] computing“

10. 10 AIP Workshop “Hands-On Globus!” 8. 11.2006 A grid or not a grid? Operating Systems + the internet?? - client-server architecture, no automated network-wide components Botnets (for spamming or DOS-Attacks)? - centrally organised Peer-to-peer networks? - function limited to data sharing Cluster? - too homogenius, specialised and centralised. Localised, different operating and security systems (but a good thing anyway) SETI project or BOINC (500.000 CPUs!)? - limited function, no cooperation. A low-end grid?

11. 11 AIP Workshop “Hands-On Globus!” 8. 11.2006 Where are we on the Hype Curve? Gardner Group (US IT consulting company) August 2005 Peak of inflated expectations Trough of disillusionment

12. 12 AIP Workshop “Hands-On Globus!” 8. 11.2006 Middleware Middleware – can you see it? middlewareGrid middleware is a set of programs: demons, shell scripts, servers, java servlets, etc. Middleware software is focused to perform specific tasks within the grid The middleware extends the operating system to allow:  dedicated servers to perform automated management tasks  a machine to become part of the grid and share resources  a user to access and use the grid. or: that‘s all very nice, but the devil is in the details

13. 13 AIP Workshop “Hands-On Globus!” 8. 11.2006 Introduction II: Globus Components

14. 14 AIP Workshop “Hands-On Globus!” 8. 11.2006 Globus history 1998: Globus V1.0; currently V4.03. Work in progress… Communities: Globus Alliance, Open Grid Forum de-facto standard in science, core elements are stable written mostly in C, web service layers Java and Python uses XML for web service documents open source „Globus is the plumbing standard of the grid, a set of blocks and tools“. Remember that grid resources are not very homogenious.

15. 15 AIP Workshop “Hands-On Globus!” 8. 11.2006 Hardware Computers, Storage, Sensors OS & Internet program execution, local user management; TCP/IP + internet protocols Middleware & Security Infrastructure User Interface (Shell / GUI) Main Architecture Elements Execution Management Data Management Monitoring Grid Infrastructure Resource Broker Information Services User Application

16. 16 AIP Workshop “Hands-On Globus!” 8. 11.2006 Main Globus Elements Resource Hardware: Resource Hardware: computers, storage, sensors provide: operating system, network access allow:program execution, local user management Globus Toolkit Middleware Data Management Data Management Execution Management Execution Management Status Information Service Status Information Service Security Infrastructure Security Infrastructure Job Monitoring Job Monitoring Job Management Job Management User Globus

17. 17 AIP Workshop “Hands-On Globus!” 8. 11.2006 The Globus Toolkit: Component Overview a) Security Infrastructure (GSI) b) Resource & Information Management c) Execution & Data Management Many services are available as command line service and additionally as web service. Some are grid-improved versions of unix commands „batch“ scripts (RLS, XML) Lots of acronyms…  see handout!

18. 18 AIP Workshop “Hands-On Globus!” 8. 11.2006 The Globus Toolkit: Component Overview About a dozen core commands for command line Many services are available in different versions: as command line service and additionally as web (augmented) service. Some are just „grid-improved“ versions of unix a commands They often use „batch“ scripts (RLS, XML), models exist Lots of acronyms…  see handout!

19. 19 AIP Workshop “Hands-On Globus!” 8. 11.2006 Security Machine and user are identified by certificates P/P-key pair and key exchange Authentication, authorisation, access policy, encryption Virtual Organisations („VO“): dedicated groups of users or resources Future: Resource usage accounting? Central certificate autority „CA“; AstroGrid-D: FZK, DFN Local registrate authority „RA“; AIP: Harry Enke Once your certificate works, you don‘t have to bother much about the security any more.

20. 20 AIP Workshop “Hands-On Globus!” 8. 11.2006 Information & monitoring MDS (monitoring and discovery service): What resources are available? What machines are there and are they useful for me? GRIS (grid resource information service): Current status of machine & job Notification & archiving Cluster: Ganglia

21. 21 AIP Workshop “Hands-On Globus!” 8. 11.2006 Execution & Data Management GRAM (Globus resource allocation management): Job submission and control GridFTP, gsiscp, globus-url-copy RFT: Reliable file transfer Data replication service (DRS), replica management (RLS): create and manage redundant replicas of datasets

22. 22 AIP Workshop “Hands-On Globus!” 8. 11.2006 [Cern grid flash] Does all that work together (at least in theory)?

23. 23 AIP Workshop “Hands-On Globus!” 8. 11.2006 What can you do on with the grid? Easy: Serial jobs that produce independend results. Only needs file management (save output, provide new input) More difficult: Interprocess communication. But network communication etc. is available.