1. OpenVZ Live Migration Jim Owens

2. Overview Review of OpenVZ Features Resource management Installation VM creation VM management Checkpointing Migration Live migration (online) Offline migration

3. OpenVZ features Open Source project of SWSoft’s Virtuozzo Run 10s, even 100s, of Linux* VMs on one box (~30 VMs per 1GB of RAM) Supports up to 64 CPUs and 64GB RAM Low virtualization overhead (1-3%) Change VM resource allocations on the fly Checkpointing Live migration *Fedora, CentOS, Gentoo, SUSE, Debian, Ubuntu, Slackware, etc.

4. Resource management Two-level disk quotas (disk blocks and inodes) Per server, per user and per group within guests Two-level CPU scheduler Configurable allocation per guest, standard Linux scheduler inside guests User bean counters ~20 configurable parameters Five associated levels: held, maxheld, barrier, limit, failcnt /proc/user_beancounters

5. Installation Install bare-bones Linux distro on hardware node (preferably Fedora, CentOS, RHEL) Set up OpenVZ yum repository Download & install OpenVZ kernel Stable version 2.6.9 Test version 2.6.18 FC5 and SUSE 10 version 2.6.16 Update /etc/sysctl.conf Reboot into new kernel Install tools: vzctl, vzpkg, vzquota Start vz service

6. OS template cache preparation Install one or more OS templates Metadata Package lists, repo location(s), scripts, keys, etc. Repocache Stores local copy of packages (can be faster) Precreated OS template cache Not recommended, as template cache cannot be updated http://openvz.org/download/template

7. VM creation # vzlist –a # vzpkgls # vzctl create 101 –ostemplate fedora-core-5-i386-default # vzctl set 101 --hostname waldo.tux.net -–save # vzctl set 101 --ipadd 192.168.10.101 –-save # vzctl set 101 --nameserver 192.168.10.1 -–save # vzctl start 101 # vzctl enter 101 # [root@waldo /]# exit exited from VE 101 # vzctl set 101 –userpasswd root:password

8. VM management # vzctl status 101 # vzctl [stop] [start] 101 # vzctl set 101 --diskspace 10000000:11000000 [-- save] # vpzcpucheck Current CPU utilization: 117662 Power of the node: 280735 # vzctl set 101 --cpuunits 8422 --cpulimit 5 //3-5%

9. VM management // Splitting the node into equal pieces # cd /etc/sysconfig/vz-scripts # vzsplit -n 20 -f vps.mytest # vzcfgvalidate ve-vps.mytest.conf-sample # vzctl create 101 --ostemplate fedora-core-5 \ --config vps.mytest # vzctl set 101 --ipadd 192.168.1.101 –save # vzctl start 101

10. Checkpointing concepts Saves all state to file and stops a running VE Checkpointed VE can be restored on same hardware node (to take snapshot?) or transferred and restored on another network- connected node For transfer, file system state on target node must be identical to system state at the moment of checkpointing. (Use rsync.)

11. Checkpoint commands Command vzctl chkpnt VEID [ --dumpfile ] Default: /vz/dump/DUMP.VEID Steps --suspend --dump [ --dumpfile ] --kill --resume (Optional, for snapshotting VE)

12. Checkpoint resume commands Command vzctl restore VEID [ --dumpfile ] Default: /vz/dump/DUMP.VEID Steps --undump [ --dumpfile ] --resume

13. Checkpoint limitations Process uses unimplemented kernel object(s) in destination node Process uses hardware facilities unavailable in destination node CPU auto-detection SSE2 or CMOV instructions Application use exotic non-virtual, user-level capabilities, such as CPU timestamps or SMP CPU IDs

14. Live migration capabilities All processes, including those started manually, continue running Open files remain accessible Network connections maintained Interactive sessions remain intact (ssh, telnet, etc) FTP downloads continue Database transactions delivered E-commerce sessions maintained

15. Live migration prerequisites Two or more servers running the core virtualization software A regular network connection between the servers

16. Live migration command vzmigrate [--online] VEID Uses rsync to transfer VE private area SSH used as transport layer; requires Root access via SSH Passphrase-less private-public keys

17. Cool facets of migration Virtualization of pids Global pids (can be changed, if required) Virtual pids (used only internally) Virtualization overhead ~0.3% incurred only for live migration Migration of network connections Move existing VE IP from source to destination node Send ARP announcement Update ARP cache of networked nodes and those performing traffic routing to VE

18. Questions and comments?