1. CSE 160 – Lecture 5 Introduction to PVM

2. PVM How PVM is structured Sending and receiving messages Getting started with the programming assignment

3. PVM Programming Model Dynamic collection of serial and parallel computers appear as single distributed memory Virtual Machine. Tasks can be dynamically spawned and killed by any other task. Any PVM task can send a message to any other. No limit to size or number of messages. Model supports fault tolerance, resource control, process control, heterogeneous networks and hosts.

4. host (one per IP address) pvmd - one PVM daemon per host pvmd How PVM is Designed libpvm - task linked to PVM library pvmds fully connected using UDP task Unix Domain Sockets inner host messages OS network interface task Shared Memory shared memory multiprocessor P0P1P2 task distributed memory MPP task internal interconnect tcp direct connect

5. Step 1 – Enrolling in PVM PVM is a user-level library and tasks must be enrolled to be addressed. myid = pvm_mytid(); printf(“ -> %x”, myid); -> 0x40001 First call to any PVM function will enroll your program PVM. Above construct is the standard practice

6. Task ID = TID PVM TIDs are very strange looking numbers. –Give host information –Node information –Type of message (unicast or multicast) –System bit for messages destined to PVMDs (as opposed to regular tasks).

7. PVM uses tid to identify pvmd, tasks, groups Fits into 32-bit integer S bit addresses pvmd, G bit forms mcast address Local part defined by each pvmd - eg. for PGON Task ID (tid) 18 bits12 bits S G host ID local part 12 bits S G host ID process node ID 11 bits7 bits 4096 hosts2048 nodeseach with

8. Popular Distributed Programming Schemes Master/Slave Master task starts all slave tasks and coordinates their work and I/O SPMD Same program executes on different pieces of the problem Functional Several programs are written. Each performs a different function in the application

9. Sending and receiving messages Send a message err = pvm_send( dest, tag); Dest = TID of destination; Tag = user-defined distinguishing integer; Receive a message Err = pvm_recv( src, tag); Src = TID of sender Tag = distinguished integer

10. Receive Wildcards A receiver may wildcard the sending tid or the matching tag. –1 is the wildcard. Any sender, specific tag pvm_recv(-1, tag); Specific sender, any message pvm_recv(src, -1); Any sender, any message pvm_recv(-1, -1);

11. PVM Examples in Package Supplied examples include: hello hello_otherPVM equivalent to hello world master slaveMaster/slave example spmdSPMD example gexampleGroup and collective ops. timing timing_slaveTiming example - comm perf hitc hitc_slaveDynamic load balance inheritb Communication context imbi gmbi Persistent messages template mhf_server mhf_tickle Message handlers Examples illustrate PVM usage and serve as templates to build your own applications. Examples include Makefile.aimk and both C and Fortran versions.

12. Starting PVM pvm [-nhostname] [hostfile] PVM console. Starts PVM or just attaches to running PVM. Recommended way to start and stop PVM 3. xpvm [hostfile] PVM Graphical Interface includes console functions, and more. Starts PVM or attaches to running PVM. pvmd [-nhostname] [hostfile] direct startup, seldom used except for debugging. Option -n useful if host has multiple network cards eg. ATM, ethernet Three ways to start PVM

13. Virtual Machine Construction via the PVM Console When you run pvm from the console –If no existing VM for this user, start a new one –If an existing VM, enroll in PVM and wait for commands. Remember! The collection of communicating PVMDs define a VM. The pvm console is “just another pvm program”. –Anything that can done from the console, can be done from your program.

14. VM Construction Cont’d VMs with one physical machine are OK, but not very interesting. –Other machines must be added from the first machine you started (eg. From whip) whip% pvm pvm> add frap host frap added A hostfile can be used to list all the hosts you want in VM. whip% pvm hostfile (start pvm on whip and add all hosts to VM listed in hostfile)

15. Some Common Startup Problems PVM needs to be to remote shell to a host to add a pvmd and extend the machine –Make sure you rsh to a machine without needing a password Old PVMD was running, but didn’t clean up properly (Pvmd already running? Error) –Look for /tmp/pvmd.uid and remove it

16. PVM Console add hostname alias conf delete hostname halt help [command] kill tid ps -a quit reset sig spawn trace version pvm> Available commands and their meaning Add host(s) to virtual machine (can list several) define/list command alias list hosts in the virtual machine delete hosts from virtual machine shut down PVM and tasks print information about commands and options kill a task list all running tasks exit console - leave PVM and tasks running kill all tasks and reset PVM send signal to a task spawn tasks (many options) set/display trace events print PVM version

17. Creating a Hostfile lo= login name for this host so=pw ask for passwd this host dx= location of pvmd ep= path to user’s executables wd= working directory to spawn tasks sp= relative speed rating bx= location of desired debugger so=ms manual pvm startup this host Simplest form is just a list of hostnames Same as initial host $PVM_ROOT/lib/pvmd ~/pvm3/bin/$PVM_ARCH $HOME 1000 (not used by PVM) $PVM_ROOT/lib/debugger use to bridge firewalls Hostfile options Default values Example: # Beowulf hostfile (comment lines start with #) * ep=/stuff/myapp wd=/stuff/myapp host1 host2 lo=tom so=pw

18. Starting tasks Use spawn in the pvm console pvm> spawn -> /bin/date pvm decides where to start task pvm> spawn -> -whip /bin/date start on host whip pvm> spawn -> –4 /bin/hostname start 4 copies Use the pvm_spawn() command to start tasks from your own program

19. PVM_ROOT PVM_ROOT – Where PVM looks for various files PVM_ARCH – architecture code for the machine –eg. LINUX, SUN4SOL2, SGI5 If you do a “spawn -> gexample”, where does it look for the executable? –$PVM_ROOT/bin/$PVM_ARCH –$HOME/bin/$PVM_ARCH Both of these paths are relative to the remote machine

20. Capturing Output When tasks are spawned, there output goes to /tmp/pvml.uid on the local machine? –Output shows up in many different files PVM answer: pvm_catchout(stdout) –Capture standard output/error from spawned (children) tasks and copy to your standard output. The ‘->’ in the console spawn command tells PVM to capture output of all tasks and display Remote output is buffered. Data will show up when buffer is full or when remote program ends.

21. First Part of Homework Assignment Write a launcher, that uses pvm_spawn, captures output, and responds to errors by killing tasks. Spawning on a particular host char *remotehost; int remtid; numt = pvm_spawn(task, task_argv, PvmTaskHost, remotehost, 1, &remtid); If spawn fails, numt != number of tasks (1); –Errors are sent back in remtid (array)

22. Pseudo code - Launcher Read nprocs and hostfile Read program name, and args Read numhosts from hostfile into a hosts array mytid = pvm_mytid(); /* enroll */ pvm_catchout(stdout); for (i = 0; i < nprocs; i++) { numt = pvm_spawn(prog, progargv, PvmTaskHost, &hosts[numhosts % i], &tids[i]); if (numt != 1) print spawning error code; for (j = 0; j < i; j ++) pvm_kill(tids[j]); pvm_exit(); exit(); } pvm_exit(); exit();

23. Messages Messages serve TWO purposes –Exchange data –Synchronize processes PVM uses the concept of message buffers –Buffers may have heterogenous and typed information in them. –Message contents are up to the user Unpacking/packing order are defined by your program. Types of packed data not included in message itself.

24. Example Packed Message buffer nitemsAoAo A1A1 A2A2 A nitems message length floats int By default, data is packed in a universal format – XDR (External Data Representation), so that the data can be read by any architecture.

25. Initializing A Message Buffer pvm_initsend(PvmDataDefault) –Initialize the current send buffer –The buffer itself is hidden. This causes no end to confusion ! Suppose you want to send a message formatted as follows: –First entry: integer K, number of strings to follow –Next K entries: string data

26. Example – Packing, sending #define TEST 101 pvm_initsend(PvmDataDefault); pvm_pkint(&K, 1, 1); /* pack one integer, note & */ for (i = 0; i < K; i ++) pvm_pkstr(&strings[I]); pvm_send(dest, TEST); Sends the packed message to dest with tag TEST;

27. Receiving the Message and Unpacking #define TEST 101 pvm_recv(-1, TEST); recv’s the packed message from anybody with tag TEST; Set the current receive buffer for unpacking. pvm_upkint(&K, 1, 1); /* find out how many strings in msg */ for (i = 0; i < K; i ++) pvm_upkstr(&strings[I]);

28. TIDs revisited TIDs have a strange assignment that makes it hard to write SPMD-style programs. PVM uses Groups to ease the mapping of TIDs to a 0 – N (for N+1, members). –Each member of a group is called an instance –Algorithms often are based on 0 – N membership PVM uses user-defined group names –look at example where N is given as a command arg PVM send/recv use TIDs as dest/src NOT instance numbers

29. Groups #define GNAME “myGroup” Read nprocs from command line mytid = pvm_mytid(); myinst = pvm_joingroup(GNAME); pvm_freezegroup(GNAME, nprocs); pvm_initsend(); pvm _pkxxx(); pvm_send(pvm_gettid(GNAME, oinst), TAG); pvm_recv(pvm_gettid(GNAME, ainst), TAG); pvm_upkxxx(); pvm_lvgroup(GNAME); pvm_exit();

30. Ring Gather - Overview 3 2 1 0 send(2, tag) 1 send(0, tag) 3 send(1, tag)2 time step 4 Print Results

31. Ring Gather – Pseudo Code int ndata = 0; myinst = pvm_joingroup(GNAME); if (myinst != nprocs – 1) pvm_recv(pvm_gettid(GNAME, myinst+1), TAG); pvm_upkint(&ndata,1,1); for (i = 1; i <= ndata; I++) pvm_upkxxx(&Data[i][0], length, 1); fill data[0][] with local information if (myinst != 0) ndata ++; pvm_pkint(&ndata,1, 1); for (i = 0; i <= ndata; i++) pvm_pkxxx(&data[i][0], length, 1); pvm_send(pvm_gettid(GNAME, myinst – 1), TAG); else /* instance 0 */ print out data entries 0 to ndata

32. Tree Gather - Overview 3 4 5 6 1 2 0 time step send(1,tag) send(2,tag) 1 send(0,tag) 2 Print Results 3

33. Tree Gather – Pseudo Code int ndata = 0; int index = 1; myinst = pvm_joingroup(GNAME); if (2*myinst + 1 < nprocs) /* “left” subtree */ pvm_recv(pvm_gettid(GNAME, 2*myinst+1), TAG); pvm_upkint(&ndata,1,1); for (i = index; i <= ndata; i++) pvm_upkxxx(&Data[i][0], length, 1); index = ndata + 1; if (2*myinst + 2 < nprocs) /* “right” subtree */ pvm_recv(pvm_gettid(GNAME, 2*myinst+2), TAG); pvm_upkint(&tmpdata,1,1); ndata += tmpdata; for (i = index; i <= ndata; i++) pvm_upkxxx(&Data[i][0], length, 1); fill data[0][] with local information

34. Tree Gather Pseudo Code Part 2 if (myinst != 0) ndata ++; pvm_pkint(&ndata,1, 1); for (i = 0; i <= ndata; i++) pvm_pkxxx(&data[i][0], length, 1); dst = (myinst – 1)/2; pvm_send(pvm_gettid(GNAME, dst), TAG); else /* instance 0 */ print out data entries 0 to ndata

35. Next Time Start on parallel algorithms and parallel programming paradigms