1. Transparent Process Migration: Design Alternatives and the Sprite Implementation Fred Douglis and John Ousterhout

2. Purposes  Utilize idle workstations  Reclaim ownership of workstations when no longer idle

3. Main Goals  Transparent  Users own their workstations  High performance  Low Complexity  Easy to use

4. Alternatives  rsh  Nichol’s Butler System  Accent  Charlotte  V  LOCUS  7up  “A Brief Survey of Systems Providing Process or Object Migration Facilities”

5. System Features  Idle hosts are plentiful  Users own their workstations  Most programs are short lived  Sprite uses kernel calls (as opposed to message passing)  Sprite provides networking mechanisms that allow for ease of implementation

6. Policies  Which processes do we migrate  Migrate only at initiation or after running a while  How do we select idle machines  Who makes the decisions

7. Possible Options  Pool of processors  No policy support (rsh)  Automated selection of idle hosts but other decisions are only partially automated

8. Selection of Idle Hosts  Each machine has a load-average daemon which monitors usage  Load-average daemon notifies central migration server when idle  Request for idle host is sent to server process

9. What determines if a host is idle?  No keyboard or mouse movement for a certain period  Fewer runnable processes than processors

10. Which idle host do we use?  The host that is idle for the longest period of time  The host with the most amount of memory

11. Transparency Definition  Behavior unaffected by migration  Appearance to rest of world unaffected by migration

12. Achieving Transparency  Changed file system so every machine sees same name space  Transfer state information from source to target  Forward kernel calls home  Ad hoc techniques for specific kernel calls such as fork

13. State Information  Virtual Memory  Open Files  Message Channels  Execution State  Other (pid, user id, working directory, signal masks and handlers, resource usage stats, references to parent/child, etc.)

14. Possible Solutions for Handling State  Transfer State  Arrange for forwarding  Ignore state and sacrifice transparency  Disallow migration for certain processes

15. Virtual Memory Mechanism  Transfer at migration (Charlotte & LOCUS)  Precopying (V)  Lazy copy (Accent)  Lazy copy … sorta (Sprite)

16. Open File Mechanism  Close file on source reopen on target  Notify file is in use on target then notify no longer in use on source  Special server code for migrating files – Source contacts target, target contacts central server, server contacts host

17. PCB and Other State Information  Home and target need PCB for a process  PCB on home contains only enough fields to locate process  PCB on target is complete  Other state is either part of PCB or can be transferred directly

18. Eviction  Load-average daemon checks for foreign processes and evicts them  Process migrated back to host  Can also occur if centralized server receives request for idle host when none are available

19. Residual Dependencies  Definition: Ongoing need for home to maintain data structures or provide functionality after migration

20. Problems with Residual Dependencies  Decrease in reliability  Decrease in performance  Increase in complexity

21. Other Problems: Interdependence of Migration and Kernel  Migration involves state manipulated by every major module in kernel  Changes in one affects the other  Solutions Migration version numbers: Disallow migration between processes with different version numbersMigration version numbers: Disallow migration between processes with different version numbers Avoid centralizing migration codeAvoid centralizing migration code

22. Algorithm  Process initiates communication with central server (opens a special file, OS passes the open operation onto server)  Process requests idle hosts  Centralized server selects idle host and sends information to the home machine  Process is signaled to make it trap into kernel  Source contacts target to confirm it is running, available for migration and has same version number

23. Algorithm Continued  Kernel on target allocates new PCB and returns a token identifier  Pre-migration routine obtains size of encapsulated data and takes necessary actions  Source kernel allocates buffer to hold encapsulated state  Call encapsulation routine to place encapsulated information into portion of buffer  Source kernel passes encapsulated state to target kernel via RPC

24. Algorithm Continued  Target begins de-encapsulation routine  Some processes also require a post- migration routine to clean up remaining state  Source kernel frees buffer and informs target to resume process execution  After process completes return host to pool of idle hosts  Close communication channel to central server

25. Practical Applications  pmake: used to parallelize make command by invoking as many commands in parallel as there are idle hosts  mig: takes as argument a shell command and executes it on an idle machine

26. Performance: Migration Overhead ActionTime/Rate Select and release idle host 36 ms Migrate null process 76 ms Transfer info for open files 9.4 ms/file Flush modified file blocks 480 Kbyte/s Flush modified pages 660 Kbyte/s Transfer exec arguments 480 Kbyte/s Fork, exec null process w/migration, wait for child to exit 81 ms Fork, exec null process locally, wait for child to exit 46 ms

27. Performance: Application Performance

28. Performance: Usage Patterns  Remote processes were 31% of execution  Evictions have been rare

29. Conclusions  Overall improvement from using idle hosts can be substantial.  Remote execution accounts for a sizeable portion of execution in Sprite. There are still a number of idle hosts.  Execution time for a process is higher if executed remotely but not noticeable by human standards.  Cost of transferring address space and flushing modified file blocks dominates migrating long running processes

30. Future Work  More applications that make use of migration  Remote execution on machines of different types  Mechanism for migration of processes that share memory  Detect when users and not just workstations are idle

31. Questions??? If you dare…