1. Transparent Process Migration for Distributed Applications in a Beowulf Cluster Mark Claypool and David Finkel Computer Science Department Worcester Polytechnic Institute Worcester, MA, USA {claypool | dfinkel}@cs.wpi.edu

2. What is a Beowulf Cluster? A cluster of PCs running Linux, Ethernet Designed for running parallel programs Standard libraries: PVM, MPI Shortcoming: The parallel libraries require explicit choice of a node to run a process on Doesn’t respond to current load on nodes

3. What is PANTS? PANTS stands for PANTS Application Node Transparency System Goal: Automatically move work from a heavily loaded (“busy”) Beowulf node to a lightly loaded (“free”) node … Sub-goal: … as transparently as possible to the user and programmer

4. Measuring Load Use the Linux /proc filesystem Version 1: Measured CPU utilization Current version: CPU utilization I/O Memory utilization Interrupts

5. Determining busy / free For each load factor: High-water / low-water marks:  Below low-water mark, busy  free  Above high-water mark, free  busy  Use of two thresholds prevents thrashing Experimentally determine thresholds for each load factor

6. Use multicasting Multicasting sends a single network message to multiple recipients Reserved set of multicast addresses Nodes listen to a multicast address when they wish to receive messages to that address (receiver-controlled) Standard feature of Ethernet protocol

7. Multicasting and PANTS (Simplified version) A multicast address is chosen for free nodes – they listen to this address A busy node writes to the free-node multicast address to locate a free node Advantage over other approaches: Busy nodes are not interrupted with messages looking for a free node

8. Multicasting and PANTS (less simplified version) To prevent receiving many replies when looking for a free node (reply implosion) : One free node is designated as leader A multicast address is reserved for leader Free nodes write to leader multicast address to notify leader that they’re free Busy nodes write to the leader multicast address to ask for a free node Possible problems with this approach …

9. The Leader Policy

10. Process Migration in PANTS New PANTS daemon on each node monitors load A new version of execve invokes prex: Examines program header to see if program is migratable If it is migratable, checks load of node If node is busy, contacts leader for free node, and transfers process via rsh

11. Results To test PANTS, compiled the Linux kernel makefile creates a large number of concurrent processes For test, started all processes on one node, allowed PANTS to migrate processes Variants: Source tree on local disk without PANTS Source tree on on NSF disk without PANTS PANTS without migration (NSF disk) PANTS with migration (NSF disk)

12. Results

13. Conclusions Automated load balancing Transparent to user and programmer Limitations: File access Interprocess communications Possible fixes: Use NSF disk DIPC (but it’s not transparent)

14. Acknowledgements We wish to thank the following WPI students who worked on various phases of the PANTS project: Jeffrey Moyer, Kevin Dickson, Chuck Homic, Bryan Villamin, Michael Szelag, David Terry, Jennifer Waite, Marc Lemaire and Jim Nichols

15. Transparent Process Migration for Distributed Applications in a Beowulf Cluster Mark Claypool and David Finkel Computer Science Department Worcester Polytechnic Institute Worcester, MA, USA {claypool | dfinkel}@cs.wpi.edu