Jaeyoung Yoon Computer Sciences Department University of Wisconsin-Madison Virtual Machine Universe in Condor

2 What is VM universe? › A job user can submit a virtual machine to Condor › Condor runs the virtual machine and sends back a result virtual machine › support VMware server and Xen

3 Big picture Submit machine Schedd Virtual Machine Shadow Execute machine Startd Starter VM GAHP

4 Benefits of VM universe › platform independence › environment independent on host machine › checkpoint › networking in a virtual machine › snapshot disk › input CDROM image

5 Snapshot disk › All modified data will be stored into snapshot disks without changing original VM disk files. › VM disk files in a shared file system can be safely shared among multiple jobs › Can reduce disk space for result and checkpoint

6 Submit description file with shared file system › universe = vm › executable = WindowsXP › vm_type = vmware › vm_memory = 256 › vm_checkpoint = TRUE › vm_networking = TRUE › vm_networking_type = dhcp › vmware_dir = /shared/windows_vm › vmware_should_transfer_files = FALSE › vmware_snapshot_disk = TRUE › initialdir = /result1 › Queue › initialdir = /result2 › Queue

7 Snapshot disk with shared file system Submit machine Execute machine 1 Shared file system Execute machine 2 /windows_vm Job 1 Snapshot Disk Job 2 Snapshot Disk /result1 /result2

8 Submit description file without shared file system › universe = vm › executable = WindowsXP › vm_type = vmware › vm_memory = 256 › vm_checkpoint = TRUE › vm_networking = TRUE › vm_networking_type = dhcp › vmware_dir = /windows_vm › vmware_should_transfer_files = TRUE › initialdir = /result1 › vmware_snapshot_disk = TRUE › Queue › initialdir = /result2 › vmware_snapshot_disk = FALSE › Queue

9 Snapshot disk without shared file system Submit machine Execute machine 1 (Job 1) Job 1 submit description... vmware_snapshot_disk = TRUE Initialdir = /result1 Job 2 submit description... vmware_snapshot_disk = FALSE Initialdir = /result2 /windows_vm Execute machine 2 (Job 2) snapshot disk

10 Submit machine Job 1 /result1 Job 2 /result2 /windows_vm Execute machine 1 (Job 1) snapshot disk Execute machine 2 (Job 2) Snapshot disk without shared file system

11 Input CDROM image › VM universe can not use input or argument parameter in a job submit description file as other universes do › With input CDROM images, a job user may run the same VM several times on different input data sets

12 Submit description file with input CDROM image › universe = vm › executable = WindowsXP › vm_type = vmware › vm_memory = 256 › vm_checkpoint = TRUE › vm_networking = TRUE › vm_networking_type = dhcp › vmware_dir = /windows_vm › vmware_should_transfer_files = FALSE › vmware_snapshot_disk = TRUE › initialdir = /result1 › vmware_cdrom_files = a.iso › Queue › initialdir = /result2 › vmware_cdrom_files = a.txt, b.txt › Queue

13 Input CDROM image Submit machine Execute machine 1 VM a.iso Job 1 submit description... vmware_cdrom_files = a.iso Job 2 submit description... vmware_cdrom_files = a.txt, b.txt Execute machine 2 VM a.txt b.txt

14 VMware VM universe › Snapshot disk › Input CDROM image › Can be used on either Linux host or Windows host

15 Xen VM universe › No support of snapshot disk  VM disk file in a shared file system can not be shared among multiple job unless it is read-only. › Input CDROM image › Can be used on only Linux host

16 Checkpoint › Periodic checkpoint and vacate checkpoint › All modified VM disk files and a file for VM memory will be transferred back to a submit machine › When snapshot disks are used, snapshot disk files and a file for VM memory will be transferred.

17 Suspend › Hard suspend: Memory being used by a VM will be released and the memory will be saved into a file › Soft suspend: Memory being used by a VM will not be released and the VM will be just paused like SIGSTOP

18 Networking issues when restarting from checkpoint › MAC and IP address for VM are also preserved when checkpointed › When restarting the checkpointed VM, MAC and IP address don’t change. › If we use NAT for VM networking, different execution machines may have different MAC and IP address of NAT gateway. › In VMware, if we install VMware tool inside VM, the tool program will automatically execute DHCP renew when a VM is restarted.

19 Future work › Support snapshot disks in Xen VM universe › For result, get only output files from VM instead of all VM files. › Support another Virtual machine program (e.g. QEMU)

20 Summary › We are testing VM universe. › Hopefully VM universe will be included in Condor 6.9.x. Questions?