1. Globus Grid Tutorial Part 2: Running Programs Across Multiple Resources

2. Using Multiple Resources 2 l Learn how to couple resources at multiple sites and use them for a single application u Required by very large applications u Also by applications that need a heterogeneous mix of resources l Learn how to run existing parallel codes under Globus u By using MPICH-G, a grid-enabled MPI l Other application examples include u SF-Express, climate models, etc. Goals of this Tutorial

3. Using Multiple Resources 3 Gas Dynamics (PPM) Brick Manager Brick Updater Brick Updater Brick Updater Task Manager Brick Manager Archiver Woodward, U Minn. l Tightly coupled CFD problem u Needs large computational power l Mask latency by overlapping communication and computation u Move data a brick at a time u Size bricks to CPU and network

4. Using Multiple Resources 4 Problems l How do we start a program running across multiple machines? u Co-allocation and scheduling u Different schedulers and security systems l What programming model should be used? u Can we run existing applications?

5. Using Multiple Resources 5 Globus Advantages l Resource management architecture provides co-allocation tools l Can mix communication methods u Nexus multimethod communication u MPI, sockets, etc. l Uniform access to local services u Security, resource management, etc. l Architecture promotes building high-level programming tools u E.g., MPICH-G, a grid-enabled MPI

6. Using Multiple Resources 6 Programming Tools: Approaches l “Hand coded” applications, combining existing tools with Globus calls u Use sockets, MPI, threads, SHM, etc. u Globus security and resource management still provide added value l Grid-enabled libraries u Manage both communication and resource management u Provide uniform programming environment across resources

7. Using Multiple Resources 7 MPICH-G l A complete implementation of the Message Passing Interface (MPI) u Passes MPICH regression test without change l MPI is the defacto standard for message- passing, parallel programs. l Enables existing MPI programs to run within a grid environment without change. l Documentation u http://www.globus.org/mpi

8. Using Multiple Resources 8 Running a Program l Goal: Run a Message Passing Interface (MPI) program on multiple computers l MPICH-G uses Globus for authentication, resource allocation, executable staging, output redirection, etc. % mpirun -np 4 my_app 4 3 2 1

9. Using Multiple Resources 9 Running an MPICH-G Program l Create a file named “machines” u A list of Globus managers and counts sp2.sdsc.edu-loadleveler 4 neptune.cacr.caltech.edu-lsf 4 jupiter.isi.edu-fork % mpirun -np 12 my_app u Creates a total of 12 tasks allocated in a round-robin fashion with “count” tasks per allocation request (sp2 4) (neptune 4) (jupiter 1) (sp2 3)

10. Using Multiple Resources 10 How MPICH-G Works l mpirun: u Locates complete globus resource manager information for specified resources (MDS) u Creates resource specification request u Calls globusrun to execute the program l Uses Nexus for communication u Delivers enhanced performance by using multiple communication protocols

11. Using Multiple Resources 11 Starting Multiple Jobs l The globusrun command: u Submits multiple simultaneous job requests u Stages executables (GASS) u Waits for termination (GRAM/DUROC) u Forwards stdout/stderr (GASS) l Convenient wrapper around several Globus services: u DUROC, GASS, GRAM, GSI, MDS

12. Using Multiple Resources 12 Globus Resource Managers l Every resource is controlled by a resource manager called a GRAM u Interfaces to local resource management system, e.g., LoadLeveler, NQE, LSF. l Every resource manage has a unique distinguished name, or DN u DN is a sequence of attribute-value pairs: /C=US/O=Globus/O=USC/OU=ISI/CN=jupiter.isi.edu-fork l The MDS stores information about each resource manage

13. Using Multiple Resources 13 Limitations of Simple mpirun l Limitations of “machines” file u Executable staging only for homogeneous sets of machines u For heterogeneous sets, executables must be placed in the same location on every machine l More general MPICH-G startup is possible u Dynamic discovery of resources u Specify name of the executable at each site u Specify location of executables and data files l Currently achieved by passing RSL string

14. Using Multiple Resources 14 Exercise 2 Introduction to MPI l Use mpirun to run an MPI program % mpirun -np 2 program l Use globus-rcp to copy files remotely % globus-rcp filename host:filename

15. Using Multiple Resources 15 Globus Components in Action DUROC fork P1 P2 GRAM LSF P1 P2 GRAM LoadLeveler P1 P2 GRAM Nexus globusrun mpirun

16. Using Multiple Resources 16 Summary l Using multiple resources located in multiple domains is a basic grid operation l Globus supports this operation via core services and high-level tools l Standard MPI programming environment provides a convenient way of building grid applications u Must be careful about configuration and latency