1. Introduction to Grid Computing with High Performance Computing

2. Objectives Training Units Introduction to Grid Computing Registration with the White Rose Grid Information Sources Outline

3. Learning Outcomes Develop a High Performance computing (HPC) application. Develop and manage applications for Computational grids Manage the execution of HPC applications over a grid computing system. Discover resources, applications and data on distributed systems. Demonstrate an appreciation of evolving Grid Technologies.

4. Sections Introduction to the White Rose Grid and grid operating systems. Application development for computational grids. Development tools and techniques for distributed high performance computing applications. Grid technologies and example grid projects

5. Assessment 3 Sets of assessment problems A mini project on grid computing with a 3000 word report Marks for the course are broken down as follows –40% from assessed problems –60% from mini project

6. Grid Computing Mini Project Grid Applications for Specific Disciplines –Design of a grid application –Implementation of a prototype –An essay on the grid technologies that will influence different disciplines and how this will be achieved.

7. Introduction to the White Rose Grid and Grid Operating Systems Introduction to Grids, Registration and Access Application management and development using the UNIX operating system. Review of White Rose Grid architecture and applications. Job Management and Scheduling. Middleware for grid computing.

8. Application Development for Computational Grids Using and developing matlab applications for the grid. Development of HPC applications using the C and Fortran Review of object orientation techniques in the development of HPC applications. Application Development using C++ Application development environments for the White Rose Grid

9. Development tools and techniques for distributed high performance computing applications. Libraries for HPC Application Development on the WRG. Performance Assessment Tools Distributed Application Development Using MPI Distributed Application Development Using Open MP

10. Grid technologies and example grid projects Introduction to Grid Services and the Open Grid Services Architecture. Development of Grid Portals Implementing a project as a grid service, The DAME project. Cluster building and Integration with Grids From semantic web to semantic grids –Distributed searching of textual databases

11. Grid Computing is New Technology Grid computing is in the pioneering age! As grids and grid technology evolves it is to be expected that the contents of course material will change.

12. Introduction to Grid Computing Grid Technologies Grid Definition Types of Grid Grid Architecture

13. Grid Technologies

14. Large scale multi site data mining and data fusion Shared virtual reality Interactive collaboration Real-time access to remote resources. Grid Technologies

15. Access Grid

16. Definition of the Grid Problem The “Grid Problem” is defined as the flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions and resources. From “Anatomy of the Grid (Enabling Scalable Virtual Organisations)” I.Foster et al. Intl. J. Supercomputer Applications 2001.

17. Characteristics of Grids Global collections of resources connected with high speed networks Supercomputers, databases, storage, instruments, immersive environments Next generation internet Emerging technology Cost effective management of high end compute and data resources

18. Grid Architecture Networks – High Bandwidth Computing Tflops Data storage Peta byte The Grid

19. Virtual Organisations

20. Evolution of Networked Computing ArchitectureClient Server DistributedWeb Applications Web Services Grid Applications Grid Services DescriptionNetwork is computer ObjectsThe webComputer is Network Virtual Organisations Globally Extended Virtual Organisations PeakLate 80’sEarly 90’sLate 90’sEarly 2kLate 2k???? ProtocolXXHTTP, JVMXML, LDAP SOAP WSDL GRIP, GRAM, GFTP, LDAP MDS,UDDI

22. The Internet and Dynamic Web Technology

23. The Web Application Model

24. Web Services Web service are OK but… They can only run on the computer Where they are put. The computer or data availability may not be fit for the task.

25. Web Services Model

26. Types of Grid Computational GridDistributed SupercomputingGrid aggregates computational resources to compute large complex problems High throughput computingGrid Schedules large numbers of loosely coupled or independent tasks On Demand ComputingMeet short term requirements for projects that cannot be cost-effectively or conveniently computed locally Data Grid Data Intensive ComputingGrid Synthesises from data managed on geographically distributed repositories Access Grid Collaborative ComputingEnhancing human-to-human interactions, e.g. collaborative design and virtual worlds

27. Grid Services Web services on Steroids! They go anywhere! Take on any task!

28. Grid Services

29. Grid Architecture APPLICATIONMolecular Modelling, Aerospace, Graphics and Visualisation APPLICATION TOOLKITSData Grid, Remote Computation, Portals, Collaborative Visualisation, Sensors GRID SERVICESProtocols, Authentication, Policy, Resource Management, Instrumentation GRID FABRICStorage, Computers, Networks, Sensors, Devices, Experiments

30. Grid Examples NASA’s Information Power Grid computing infrastructure connecting the resources of several of its R&D Laboratories forming the Information Power Grid or IPG High Energy Physics E Data Grid The DataGrid Project is providing the solution for storing and processing data generated by the LHC at CERN.

31. Grid and Web Service Demonstrations SETI@home The Dame Portal –Requires valid e-Science Certificate Virtual Observatory European Data Grid Demonstrators OpenGIS Web Service Matlab data Mining Service

32. Statistics for SETI at Home (16/10/2003) TotalLast 24 Hours Users 47103991534 Results received 1073252223 1377476 Total CPU time 1655366.998 years1347.953 years Floating Point Operations 3.769869e+215.372156e+18 (62.18 TeraFLOPs/sec) Average CPU time per work unit 13 hr 30 min 40.6 sec 8 hr 34 min 20.0 sec

33. SETI@home’s Most Promising Candidates

34. The DAME Portal Distributed Aircraft Maintenance Environment Portal runs engine vibration analysis tools on selected sets of engine performance data. http://iri02.leeds.ac.uk:9080/damexto/damexto

36. Data Grid Portal Demonstrators –Medical Imaging –Bioinformatics http://edg-wp10.healthgrid.org

38. Visit the Virtual Sky Portal http://virtualsky.org

39. OpenGIS Web Service http://www.opengis.org/resources/?page=demos Standards based Data Connections –Forestry information –Satellite data Landsat 7, ESA Global imagery data –Atlas data, Roads, boundaries, populations, water areas.. Based on Web services enables easy integration with OGSA based Grid Services

41. e-Science Projects Distributed Aircraft Maintenance Environment Distributed Collaborative Visualisation and problem solving environments Clinical and Biomedical projects A co-operative clinical e-Science Framework Biosim GRID Decision making in the health sector Lots of projects Here are a few!!

42. White Rose Grid Registration Conditions of Use Becoming a White Rose Grid user Obtaining an e-Science certificate

43. Conditions of Use White Rose Grid users must abide by the conditions pertaining to the individual systems at York, Leeds and Sheffield. –http://www.shef.ac.uk/calendar/compregs.html –http://www.leeds.ac.uk/iss/rules/ISShtml –http://www.wrg.york.ac.uk

44. Becoming a White Rose Grid User 1.Register for an account on your local system 2.Apply for White Rose Grid resources 3.Obtain an e-Science certificate

45. Why obtain an x509v3 Certificate Enables secure single sign on to the White Rose Grid Use portals e.g. the DAME portal Use Globus to access WRG compute nodes

46. Register for an Account on Your Local System InstitutionRegistration Details The University of Sheffield, Titania Email: registration@sheffield.ac.uk Web: http://www.shef.ac.uk/cics/support/userreg/r egistration.html The University of Leeds, Maxima,Snowdon http://www.leeds.ac.uk/iss/helpdesk/ usernames.html The University of York, Pascali http://www.wrg.york.ac.uk/access.html

47. Application for White Rose Grid Resources Details at http://www.shef.ac.uk/wrgrid/access Complete the Application form “Application for a Username and Resources on the White Rose Grid” Forward application to local member of the White Rose Grid executive and obtain Authorisation. Await confirmation of registration

48. Accessing the White Rose Grid From a UNIX account node access using ssh or XServer application (e.g. Exceed Using Globus with e-Science Certificate White Rose Grid Portal (requires e-Science certificate)

49. Accessing the White Rose Grid Resources –titan00.shef.ac.uk (10 Sun V880 Sun Fire Servers) –pascali.york.ac.uk (1 Sun V880 Sun Fire Server) –maxima.leeds.ac.uk (5 Sun V880 Sun Fire Servers) –snowdon.leeds.ac.uk (256 node Intel Beowulf cluster) From a UNIX account node access using ssh –ssh –l wrsmyname –X [node name].ac.uk Allows X applications from maxima and pascali

50. References The White Rose Grid http://www.wrgrid.org.uk http://www.shef.ac.uk/wrgrid/trainingresources http://www.shef.ac.uk/wrgrid/access Grid Support Centre http://www.grid-support.ac.ukhttp://www.grid-support.ac.uk National e-Science Centre http://umbriel.dcs.gla.ac.uk/NeSChttp://umbriel.dcs.gla.ac.uk/NeSC UK DTI e-Science programme http://www.escience-grid.org Global grid forum http://www.ggf.org

51. Registration Register with the White Rose Grid Obtain an e-Science Certificate Go to the link http://www.shef.ac.uk/wrgrid/access/index.html

52. Grid Computing References The Grid: Computing Without Bounds –Ian Foster, Scientific American, April 2003. “The Anatomy of the Grid” –http://www.globus.org/research/papers/anatomy.pdf Grid Services – “The Physiology of the Grid” –http://www.gridforum.org/ogsi-wg/drafts/ogsa_draft2.9_2002-06- 22.pdf Research Agenda for the Semantic Grid –http://www.semanticgrid.org/v1.9/semgrid.pdf

53. Demonstrators To Try http://setiathome.ssl.berkeley.edu/ http://virtualsky.org http://edg-wp10.healthgrid.org http://iri02.leeds.ac.uk:9080/damexto/damexto http://www.opengis.org/resources/?page=demos http://eepc269.eng.ohio-state.edu/matlab/