1. XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop

2. Copyright 2013 Citrix Page 2 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Table of Contents MotivationPage 3 WarningPage 4 First StepsPage 5 UNetbootin OptionsPage 6 UNetbootin ExecutionPage 7 Boot the XCE Computer into Ubuntu LinuxPage 8 Ubuntu Desktop and Command TerminalPage 9 Ubuntu NetworkingPage 10 Enable SSH AccessPage 11 Mount the XCE Log Volume (Part 1)Page 12 Mount the XCE Log Volume (Part 2a)Page 13 Mount the XCE Log Volume (Part 2b)Page 14 Mount the XCE Log Volume (Part 2c)Page 15 Create a Log File ArchivePage 16 Upload the Log Archive to ShareFilePage 17 Shutdown Ubuntu Live DesktopPage 18 Appendix A: Engine Boot ParametersPage 19

3. Copyright 2013 Citrix Page 3 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Motivation There are two main ways to get diagnostics from XenClient Enterprise Engine: A problem report may be submitted directly from the Engine. This will upload a standard diagnostics package to Citrix technical support. Engine diagnostics can also be requested through the Synchronizer console. These mechanisms are usually reliable and convenient, but sometimes they won't work: If the Engine is experiencing network connectivity issues and can't upload diagnostics to Citrix or to the Synchronizer server. If the Engine is in a state where it won't boot properly. For problems installing the Engine software onto a computer. An alternate method for extracting Engine log files from a computer is to boot the computer into a Linux Live session, then mount the disk volume containing the Engine log files. The log files can analyzed in place, copied to USB or network storage, or uploaded to ShareFile or a different network location.

4. Copyright 2013 Citrix Page 4 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Warning Extreme caution should be used when following these procedures, especially if you are not familiar with root-level access in Linux or Unix environments. The Linux command-line is very powerful but also potentially dangerous. Incorrect commands can cause severe problems and there are usually no warning or confirmation messages before carrying out destructive operations. If you have any questions or uncertainty about these procedures, please contact Citrix Technical Support for assistance. Citrix cannot be responsible for any problems that may arise as a result of incorrect Linux usage, including damage to the Engine installation or deployed VMs.

5. Copyright 2013 Citrix Page 5 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop What Is Needed: Direct physical access to the XenClient Enterprise computer. A USB stick (2 GB or larger) that can be formatted. A separate Windows computer to run the UNetbootin program. The XenClient Enterprise computer must be connected to a network with Internet access. Format the USB Stick: In Windows explorer, format the USB stick with a new FAT32 file system. This will destroy all data on the USB stick! Don’t skip this step. UNetbootin works best with newly formatted USB sticks. Download and Run UNetbootin: Download from http://unetbootin.sourceforge.net/http://unetbootin.sourceforge.net/ UNetbootin is a simple executable file with a name like unetbootin-windows-585.exe. Download the file to a convenient location and run it. First Steps

6. Copyright 2013 Citrix Page 6 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop UNetbootin Options Select Ubuntu 13.04_Live. This should be set to USB Drive. Make sure the correct drive letter is set. This should be a newly-formatted USB stick.

7. Copyright 2013 Citrix Page 7 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop UNetbootin Execution UNetbootin will download the Linux Live distribution as an ISO file. Linux Live distributions are typically a few hundred MB in size and may take a while to download. After the download is complete, UNetbootin will copy the contents of the ISO file to the USB stick and configure the USB stick as a bootable device. This process will take a few minutes to complete. When UNetbootin is done, it will offer the option to reboot or exit. Do not reboot! Instead, exit Unetbootin, eject the USB stick in Windows explorer, and remove it from the Windows computer.

8. Copyright 2013 Citrix Page 8 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Boot the XCE Computer into Ubuntu Linux After UNetbootin is finished, eject the USB stick from Windows then transfer it to the XenClient Enterprise computer. Boot the XenClient Enterprise computer from the USB stick. When the computer boots, choose the Try Ubuntu Without Installing option.

9. Copyright 2013 Citrix Page 9 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop The computer should boot into the Ubuntu Desktop. To open a command terminal, click the icon indicated below and search for terminal. Root access is required for some of the commands that follow, so run sudo su in the terminal to become the root user. Ubuntu Desktop and Command Terminal

10. Copyright 2013 Citrix Page 10 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop To configure networking or view networking status, click the networking icon at the top right of the Ubuntu Desktop. Ubuntu Networking To view the current IP address, click the networking icon and select Connection Information:

11. Copyright 2013 Citrix Page 11 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Enable SSH Access CommandDescription whoami Confirm that you are the root user. If not, run sudo su to become the root user. passwd Set a password for the root user. This password will be used when connecting to Ubuntu Live Desktop via SSH. apt-get install openssh-server Install the SSH server package. Internet access is required to download the package from the Ubuntu software repository. start ssh Start the SSH service. The service is usually started automatically after the SSH server package is installed. status ssh Verify the SSH service is started. ssh root@localhost Verify SSH access (then exit the SSH session). SSH access is not enabled by default in Ubuntu Live Desktop. Run these commands in the Ubuntu terminal to enable SSH access. If these commands are successful, external SSH clients should be able to connect to Ubuntu Live Desktop with the user root and the password set with the passwd command.

12. Copyright 2013 Citrix Page 12 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Mount the XCE Log Volume (Part 1) CommandDescription apt-get install lvm2 Installs the LVM2 package. Internet access is required to download the package from the Ubuntu software repository. Only necessary if LVM2 is not already installed. vgchange -a y Discovers logical volumes (LVs) on the hard disk. Seven LVs should be discovered. lvs Lists LVs. There should be one volume group named NxVG-UUID (each computer has a different UUID). The volume group should contain seven LVs: NxDisk1 (boot volume) NxDisk2 (boot volume) NxDisk5 (root volume) NxDisk6 (root volume) NxDisk7 (swap volume) NxDisk8 (log volume) NxDisk9 (Engine repository volume) ls /dev/mapper/ Lists LV devices. There should be a device entry for each of the seven LVs in the NxVG-UUID volume group. mkdir /mnt/8 Creates a new directory that will be used as a mount point for the log volume. These commands are for encrypted or unencrypted computers.

13. Copyright 2013 Citrix Page 13 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop This is for unencrypted computers. For encrypted computers, proceed to Part 2b. Mount the XCE Log Volume (Part 2a) CommandDescription mount -o ro /dev/mapper/NxVG*8 /mnt/8 Mounts the log volume to the mount point /mnt/8. cd /mnt/8 Change directory to the mounted log volume. ls -l List files in the log volume. If these commands are successful the Engine log volume should be mounted to /mnt/8 (read-only).

14. Copyright 2013 Citrix Page 14 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Mount the XCE Log Volume (Part 2b) CommandDescription cryptsetup luksOpen /dev/mapper/NxVG*5 nx5 Unlocks encrypted volume 5. The encryption password or unlock code is required. ls -l /dev/mapper/nx5 Verify this device entry was created. mkdir /mnt/5 Create a mount point for unlocked volume 5. mount -o ro /dev/mapper/nx5 /mnt/5 Mount unlocked volume 5. ls -l /mnt/5/etc/keys/nxtop.key The key file should exist. cryptsetup create nx8 /dev/mapper/NxVG*8 --key-file=/mnt/5/etc/keys/nxtop.key This is one long command. Unlock encrypted volume 8 using the key from volume 5. ls -l /dev/mapper/nx8 Verify this device entry was created. mount -o ro /dev/mapper/nx8 /mnt/8 Mount the unlocked log volume. cd /mnt/8 Change directory into the mounted log volume. ls -l List files in the mounted log volume. This is for encrypted computers. For unencrypted computers, use Part 2a instead. If these commands are successful the Engine log volume should be mounted to /mnt/8 (read-only).

15. Copyright 2013 Citrix Page 15 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Mount the XCE Log Volume (Part 2c) CommandDescription cryptsetup luksOpen /dev/mapper/NxVG*6 nx6 Unlocks encrypted volume 6. The encryption password or unlock code is required. ls -l /dev/mapper/nx6 Verify this device entry was created. mkdir /mnt/6 Create a mount point for unlocked volume 6. mount -o ro /dev/mapper/nx6 /mnt/6 Mount unlocked volume 6. ls -l /mnt/6/etc/keys/nxtop.key The key file should exist. cryptsetup create nx8 /dev/mapper/NxVG*8 --key-file=/mnt/6/etc/keys/nxtop.key This is one long command. Unlock encrypted volume 8 using the key from volume 6. ls -l /dev/mapper/nx8 Verify this device entry was created. mount -o ro /dev/mapper/nx8 /mnt/8 Mount the unlocked log volume. cd /mnt/8 Change directory into the mounted log volume. ls -l List files in the mounted log volume. This is for encrypted computers if Part 2b doesn't work. Depending on the Engine upgrade history, the key to unlock the log volume may be in volume 6 instead of volume 5. If these commands are successful the Engine log volume should be mounted to /mnt/8 (read-only).

16. Copyright 2013 Citrix Page 16 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Create a Log File Archive CommandDescription cd /mnt/8 Change directory into the mounted Engine log volume. zip -r /tmp/logs.zip * Zip up the contents of the mounted Engine log volume. ls -l /tmp/logs.zip Verify the zip file was created. This is for encrypted or unencrypted computers. After the Engine log volume is mounted, run these commands to create a zip file archive of the log volume contents. The zip file archive may be copied to USB or network storage, or uploaded to ShareFile or another network location.

17. Copyright 2013 Citrix Page 17 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Upload the Log Archive to ShareFile Select the logs.zip file and click Open to complete the ShareFile upload process. When the file is uploaded, notify Citrix that the file should be available. In the Upload Files section, click the icon to select a file for upload. Then select File System and the tmp folder. Launch FireFox from the Ubuntu desktop and enter the ShareFile upload URL.

18. Copyright 2013 Citrix Page 18 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Shutdown Ubuntu Live Desktop When finished extracting log files from the Engine log volume, shutdown the Ubuntu Live Desktop session as shown. Don't forget to remove the USB stick before rebooting the computer back into the XenClient Enterprise Engine.

19. Appendix A: Engine Boot Parameters

20. Copyright 2013 Citrix Page 20 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Mounting Boot Volumes in Linux Live Desktop It may be useful to mount the Engine boot volumes. Usually this is done to change boot parameters for the hypervisor (Xen) or dom0 (commonly referred to as "the Engine" or simply "Linux"). There are two boot volumes (volume 1 and volume 2) used for the "current" and "previous" Engine versions. For Initial Engine install, volume 1 is the "current" version (volume 2 is empty). For the first Engine upgrade, volume 2 is "current" (volume 1 is "previous"). For the second Engine upgrade, volume 1 is "current" (volume 2 is "previous"). Etc. Use the procedures outlined in this document to boot the Engine computer into a Linux Live Desktop session, and open a command-line terminal with root privileges. Then run these commands to mount the boot volumes: apt-get install lvm2 vgchange -a y mkdir /mnt/1 mkdir /mnt/2 mount /dev/mapper/NxVG*1 /mnt/1 mount /dev/mapper/NxVG*2 /mnt/2 This will mount the boot volumes read/write, not read-only, so the contents of the volumes may be modified. Best practice is to make a backup copy of any file before changing it. Incorrect changes to files in the boot volumes may render the Engine unbootable. Boot parameters for the Engine are defined in the following GRUB configuration files. It can be difficult to tell which is for the "current" or "previous" Engine version, so usual practice is to edit both files. /mnt/1/grub/grub.cfg /mnt/2/grub/grub.cfg

21. Copyright 2013 Citrix Page 21 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Setting Boot Parameters in grub.cfg In the grub.cfg file, look for a section starting with #DEFAULT_START. It should look like this: #DEFAULT_START menuentry "Boot Entry 1: XenClient Enterprise Engine 5.0.3 ${debug_status}" { saved_entry=0 save_default_entry set root=(NxVG-210d63b6-3872-4439-aa10-f8bbe4c045d7-NxDisk1) multiboot /xen.gz [hypervisor boot parameters] module /vmlinuz-3.8.13-orc [dom0 boot parameters] module /initrd.img-3.8.13-orc } #DEFAULT_END This is where the hypervisor (Xen) and Engine (dom0) boot parameters are defined. In this example, the actual boot parameters are omitted to avoid confusion caused by line wrapping (these lines can be long). Green line (starting with "multiboot /xen.gz"): Boot parameters for the hypervisor (Xen). Orange line (starting with "module /vmlinuz-3.8.13-orc"): Boot parameters for dom0 (often referred to as "the Engine" or simply "Linux"). To add a boot parameter: Append the new boot parameter to the end of the correct line (not the beginning!). Boot parameters should be separated by a single space. Do not create new lines or reformat any section of grub.cfg (it is OK if the lines grow long).

22. Copyright 2013 Citrix Page 22 XenClient Enterprise 5.0 Extracting Engine Logs with Ubuntu Live Desktop Example Boot Parameter Settings Enable Verbose Boot This will enable additional messages during Engine boot that may aid debugging. Hypervisor boot parameter changes: None needed Dom0 boot parameter changes: Remove splash Remove quiet Enable Serial Console This will redirect console output to the serial port for computers with an on-board serial port (not a USB serial adapter). A different computer could then establish a serial-to-serial connection using a null modem cable and a serial terminal client program such as minicom, hyperterm, or putty. Hypervisor boot parameter changes: Set com1=115200,8n1 (may be set to NULL or something else already, change it) Set console=com1 Dom0 boot parameter changes: Remove splash Remove quiet Set xencons=tty Set console=hvc0