1. 1 Open Source Virtualization Hacks Niel M. Bornstein niel@bornstein.atlanta.ga.us O'Reilly Open Source Convention 2008

2. 2 Agenda The Abstract The Technology The Hacks

3. 3 The Abstract Open source virtualization systems use the usual suite of tools and languages—can anyone out there say Python and XML? In this session, you’ll see some ways to build a management system, using the tools you already know, to do some wicked things with virtual machines. Along the way you’ll learn about the Open Virtual Machine Format (OVF) and some of the other emerging standards that are helping modern data centers run.

4. 4 The Technology This presentation will make use of the Xen hypervisor on openSUSE 11.0, using libvirt's python bindings. The hacks and methods presented should be equally applicable to other operating systems and hypervisors supported by libvirt.

5. 5 The Technology – openSUSE 11.0 “The openSUSE project is a community program sponsored by Novell. “Promoting the use of Linux everywhere, openSUSE.org provides free, easy access to the world's most usable Linux distribution, openSUSE. “The openSUSE project gives Linux developers and enthusiasts everything they need to get started with Linux.” http://software.opensuse.org/

6. 6 The Technology – Xen 3.2.1 “The Xen ® hypervisor, the powerful open source industry standard for virtualization, offers a powerful, efficient, and secure feature set for virtualization of x86, x86_64, IA64, PowerPC, and other CPU architectures. “It supports a wide range of guest operating systems including Windows ®, Linux ®, Solaris ®, and various versions of the BSD operating systems.” http://xen.org/

7. 7 The Technology – libvirt 0.4.0 “A toolkit to interact with the virtualization capabilities of recent versions of Linux (and other OSes) “Free software available under the GNU Lesser General Public License “A long term stable C API “A set of bindings for common languages “A CIM provider for the DMTF virtualization schema” http://libvirt.org/python.html

8. 8 The Technology – Python 2.5.2 “Python is a dynamic object-oriented programming language that can be used for many kinds of software development.” http://python.org

9. 9 The Hacks What’s a Hack? “Hacks are tools, tips, and tricks that help users solve problems. They are aimed at intermediate-level power users and scripters.” ✔ http://oreilly.com/hacks/ http://oreilly.com/hacks/ Eleven Hacks in Three Categories Exploration Basic Tasks Advanced Tasks

10. 10 Hack #1 – The xm Command Most Linux geeks like to use the command line. The Xen hypervisor provides a command line management user interface. The xm command allows you to do most common single-host management tasks. NAME xm - Xen management user interface SYNOPSIS xm [args] DESCRIPTION The xm program is the main interface for managing Xen guest domains. The program can be used to create, pause, and shutdown domains. It can also be used to list current domains, enable or pin VCPUs, and attach or detach virtual block devices. The basic structure of every xm command is almost always: xm [OPTIONS]...

11. 11 Hack #1 – The xm Command (cont'd) The command line is very useful, but can be tricky to do some tasks. For example, to install a new VM from scratch, you first need to define it using configuration files. Only then can you use the xm create command to start the install process. ✔ More about the configuration file format, and ways to edit it, later. Because it's a complex process, virt-manager makes it much easier to build a new VM.

12. 12 Hack #2 – The virsh Command virsh is another command line tool, using libvirt to provide access to a variety of hypervisors in a generic manner. More about libvirt later! NAME virsh - management user interface SYNOPSIS virsh [args] DESCRIPTION The virsh program is the main interface for managing virsh guest domains. The program can be used to create, pause, and shutdown domains. It can also be used to list current domains. Libvirt is a C toolkit to interact with the virtualization capabilities of recent ver ‐ sions of Linux (and other OSes). It is free software available under the GNU Lesser General Public License. Virtualization of the Linux Operating System means the ability to run multiple instances of Operat ‐ ing Systems concurrently on a single hardware system where the basic resources are driven by a Linux instance. The library aim at providing long term stable C API initially for the Xen paravirtualization but should be able to integrate other virtualization mechanisms, it cur ‐ rently also support QEmu and KVM....

13. 13 Hack #2 – The virsh Command (cont'd) The entire libvirt API is available from the virsh command. chonju:~ # virsh Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # dominfo 1 Id: 1 Name: opensuse11 UUID: 60c451d0-9009-2bd6-1cfb-f71e9ec6926d OS Type: linux State: blocked CPU(s): 1 CPU time: 3.3s Max memory: 393216 kB Used memory: 393216 kB virsh # shutdown 1 Domain 1 is being shutdown virsh #

14. 14 Hack #3 – virt-manager virt-manager is a project hosted by Red Hat designed to provide a graphical UI for managing virtual machines. It is written in Python with Glade and GTK+ and uses libvirt to access a variety of virtualization systems. It is included in most modern Linux distributions.

15. 15 Hack #3 – virt-manager (cont'd) You can even view multiple virtualization technologies through a single interface. Select the File -> Open Connection menu command and you will see that you can also connect to a QEMU hypervisor in the same interface, as well as making remote connections to other machines.

16. 16 Hack #3 – virt-manager (cont'd) Building a new VM is also simple. Click on the New button to bring up the vm-install process, which can also be launched directly from the command line. The vm-install wizard will present you with all the options you need to configure a new VM, and it's much easier than manually editing configuration files. Tip: To build a VM with an ISO install source, loop mount the iso first: ✔ losetup -f /path/to/iso

17. 17 Hack #4 – libvirt + Python After the command line and the GUI tool, the next step is to build your own tools programmatically. Using libvirt, you can manage a variety of virtualization systems from a variety of programming languages. For these examples, we'll use Xen and Python, because, well, it's really easy. After the first example, I will omit error checking for brevity!

18. 18 Hack #4 – libvirt + Python (cont'd) The following sample code comes from the libvirt website: #!/usr/bin/python import libvirt import sys conn = libvirt.openReadOnly(None) if conn == None: print 'Failed to open connection to the hypervisor' sys.exit(1) try: dom0 = conn.lookupByName("Domain-0") except: print 'Failed to find the main domain' sys.exit(1) print "Domain 0: id %d running %s" % (dom0.ID(), dom0.OSType()) print dom0.info()

19. 19 Hack #5 – Query the Hypervisor's Capabilities Before taking any action using libvirt, it's a good idea to get an idea of what the hypervisor is capable of doing. libvirt provides a couple of convenient methods for doing this: virConnect.getCapabilities() returns information in XML format about the hypervisor. virConnect.getInfo() returns a python dict of information about the physical node.

20. 20 Hack #5 – Query the Hypervisor's Capabilities (cont'd) #!/usr/bin/python import libvirt conn = libvirt.open(None) print conn.getCapabilities() info = conn.getInfo() print "CPU model: %s" % info[0] print "memory: %d kB" % info[1] print "# of CPUs: %d" % info[2] print "CPU freq: %d MHz" % info[3] print "# of NUMA cell: %d" % info[4] print "# of CPU sockets: %d" % info[5] print "# of cores per socket: %d" % info[6] print "# of threads per core: %d" % info[7]

21. 21 Hack #6 – Start/Stop a VM Now that we know what the hypervisor is capable of, we can start a VM instance. We've already seen how to do this with the xm command and virt-manager; now we'll do it from a Python script. #!/usr/bin/python import sys, libvirt vmname = sys.argv[1] conn = libvirt.open(None) dom = conn.lookupByName(vmname) dom.create() dom = conn.lookupByName(vmname) print "%d started" % dom.ID()

22. 22 Hack #6 – Start/Stop a VM (cont'd) Similarly, we can stop a VM, no matter whether it was started through libvirt, the xm command, or virt-manager. #!/usr/bin/python import sys, libvirt vmid = int(sys.argv[1]) conn = libvirt.open(None) dom = conn.lookupByID(vmid) dom.shutdown()

23. 23 Hack #7 – Migrate a VM In real life, you may occasionally need to move a running VM from one physical host to another. This is referred to as “migrating” the VM. I can't demonstrate doing this on a single laptop, but I can show you some code that should work!

24. 24 Hack #7 – Migrate a VM (cont'd) #!/usr/bin/python import sys, getpass, libvirt vmname = sys.argv[1] uri = sys.argv[2] mydata = "" def getCredentials(credentials, data): for credential in credentials: print credential[1] + ":", if credential[0] == libvirt.VIR_CRED_AUTHNAME: data = sys.stdin.readline() data = data[0:len(data)-1] credential[4] = data elif credential[0] == libvirt.VIR_CRED_PASSPHRASE: credential[4] = getpass("") else: return -1 return 0 flags = [libvirt.VIR_CRED_AUTHNAME,libvirt.VIR_CRED_PASSPHRASE] auth = [flags,getCredentials,mydata] localconn = libvirt.open(None) dom = localconn.lookupByName(vmname) remoteconn = libvirt.openAuth(None,auth,0) dom.migrate(remoteconn,libvirt.VIR_MIGRATE_LIVE,None,uri,0)

25. 25 Hack #7 – Migrate a VM (cont'd) In reality, it ends up being much easier to use the xm migrate command to do this. If I had to make a recommendation, I'd say that unless you're building a complete management system, you should just use the xm or virsh tools to migrate your VMs: xm migrate --live domain newhost virsh migrate --live domain newhost In these commands, the domain parameter is either the name or id of the domain, and the newhost parameter is the URI of the new host.

26. 26 Hack #7 – Migrate a VM (cont'd) The /etc/xen/xend-config.sxp file that ships with Xen does not allow migrations. Make the following changes to turn migration on: Uncomment this line and change no to yes: ✔ #(xend-relocation-server no) Uncomment this line: ✔ #(xend-relocation-port 8002) Uncomment this line: ✔ #(xend-relocation-address '') Customize this line for your security needs: ✔ (xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$') Then restart xend with the command rcxend restart

27. 27 Hack #8 – Edit a VM Configuration A VM can be thought of as a combination of configuration information and a disk image. You can change the VM configuration when a VM instance is not running by editing the configuration file (in Xen, that's the file in /etc/xen/vm/vmname ).

28. 28 Hack #8 – Edit a VM Configuration (cont'd) It's easy enough to modify a config by hand, once you understand the format. name="opensuse11" uuid="60c451d0-9009-2bd6-1cfb-f71e9ec6926d" memory=384 vcpus=1 on_poweroff="destroy" on_reboot="restart" on_crash="destroy" localtime=0 keymap="en-us" builder="linux" bootloader="/usr/lib/xen/boot/domUloader.py" bootargs="--entry=xvda2:/boot/vmlinuz-xen,/boot/initrd-xen" extra=" " disk=[ 'file:/var/lib/xen/images/opensuse11/disk0,xvda,w', ] vif=[ 'mac=00:16:3e:49:b8:b2', ] vfb=['type=vnc,vncunused=1']

29. 29 Hack #8 – Edit a VM Configuration (cont'd) But why do it by hand when we've got Python? #!/usr/bin/python import sys, os original = sys.argv[1] new = sys.argv[2] f = file(original, "rb") lines = f.readlines() dict = {} for line in lines: pieces = line.partition("=") dict[pieces[0]] = eval(pieces[2]) for key in dict.keys(): if key == "name": dict[key] = new elif key == "disk": disks = dict[key] dict[key] = [] for disk in disks: dict[key].append(disk.replace(os.path.basename(original),new)) elif key == "vif": dict[key] = None elif key == "uuid": dict[key] = None if dict[key] != None: print "%s=%s" % ( key, repr(dict[key]) )

30. 30 Hack #8 – Edit a VM Configuration (cont'd) You can change the configuration of a running VM instance using the xm or virsh commands, or using libvirt. #!/usr/bin/python import sys, libvirt vmname = sys.argv[1] maxMemory = int(sys.argv[2]) conn = libvirt.open(None) dom = conn.lookupByName(vmname) print dom.maxMemory() dom.create() dom.setMaxMemory(maxMemory) dom = conn.lookupByName(vmname) print dom.maxMemory()

31. 31 Hack #9 – Edit a VM Image Editing a VM image file is called “cracking it open”. The image file needs to be mounted as a loopback filesystem. The easiest way to crack open a Xen image is to loop mount it using lomount. lomount -diskimage disk0 -partition 2 /mnt Bear in mind that the image file will almost certainly have more than one partition. This method will not work with LVM disks!

32. 32 Hack #10 – Clone a VM Now we have edited a VM configuration and image. Cloning a VM is as simple as copying the configuration and image to a new location, then editing the configuration and “personalizing” the image. You just saw how our friend Python can easily read in a configuration file and make the necessary changes.

33. 33 Hack #10 – Clone a VM (cont'd) Just one little problem: my disk image is 8 gigs. That takes a while to copy The solution: Copy-on-Write. Use the QEMU CoW (qcow) format Convert your raw image to qcow using img2qcow img2qcow destination source There is also a corresponding qcow2raw

34. 34 Hack #11 – Create an Appliance Appliances can be defined using Open Virtual Machine Format (OVF), a standard proposed by VMware under the auspices of the Distributed Management Task Force (DMTF). OVF is: A packaging format for software applications ✔ Bundle together your VMs for a self-contained application A transport mechanism for virtual machine templates ✔ Distribute your OVFs as images which must be installed before they can be run

35. 35 Hack #11 – Create an Appliance (cont'd) OVF is not: An efficient execution environment ✔ VMs must be installed before they can be run A format requiring a hypervisor ✔ By itself, OVF does not have any dependence on a hypervisor

36. 36 Hack #11 – Create an Appliance (cont'd) The OVF file is a zip file containing An XML file (.ovf ) with schema http://schemas.dtmf.org/ovf/envelope http://schemas.dtmf.org/ovf/envelope ✔ May refer to external URIs for actual VM images ✔ May contain virtual hardware resource descriptions May contain a manifest file (.mf ) May contain a certificate file (.cert )

37. 37 Hack #11 – Create an Appliance (cont'd) Today, there is no OVF tool for Xen, only VMware! IBM is sponsoring an Open OVF project http://www.xen.org/files/xensummitboston08/open-ovf-proposal.pdf

38. 38 Hack #11 – Create an Appliance (cont'd) For another take on appliances, check out Nat Friedman's talk: The Future of Linux is Software Appliances ✔ Friday 10:45 in Portland 255

39. 39 Q&A