Copyright 2015 by Assured Information Security, Inc. Created by Martin Osterloh <osterlohm@ainfosec.com>. This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Contributions by: Ross Philipson <philipsonr@ainfosec.com>

Initial Setup

The following instructions assume a few things. For the build environment:

You have a build tree of master (6.0) OpenXT.
The instruction are relative to the base of you build tree i.e. where you cloned openxt.git.
You have installed a version of OpenXT on a target system with Measured Launch disabled that will be compatible with stuffs built in your build tree (preferably an install from the ISO of your build).
You have ssh access to the target system and a serial console. The latter will be very useful when we disable the NDVM.
Clone our working repo someplace that makes sense: https://github.com/rossphilipson/travail

Get a VHD:

Start with a Debian x64 VM. We are using Debian Wheezy x64 named /storage/disks/debian-7.8-x64-clean.vhd in the Debian xl conf.
You can make one by installing a VM on an OpenXT system (no disk encryption) or copy over an existing VHD.

Building

For these steps the target system will need to be up with network access. SELinux should also be disabled to allow copying files to the /storage dir. Do this:

$ nr

$ setenforce 0

libxl and blktap2

First we will make sure that libxl and blktap2 are compiled as part of the xen-tools package. In order to do so, we edit the file build/repos/xenclient-oe/recipes-extended/xen/xen-tools.bb. Get the patch file here: travail/blob/master/openxt/xl/xen-tools-build-xl-blktap2.patch. Do this:

$ cd build/repos/xenclient-oe

$ patch -p1 < xen-tools-build-xl-blktap2.patch

The libxl stuff needs to be patched before building the bits. Get the patch file here: travail/blob/master/openxt/xl/xl-fix-misc-issues.patch. Do this:

$ cd build/tmp-glibc/work/core2-32-oe-linux/xen-tools/4.3.4-r0/xen-4.3.4

$ patch -p1 < xl-fix-misc-issues.patch

Next we want to rebuild xen-tools and produce the new libs and bins for libxl and blktap2. Back in the build dir do this (note the capitol C):

$ ./bb xen-tools -C compile

Once the build completes, you should now have all the binaries you need. Tar them up:

$ cd build/tmp-glibc/work/core2-32-oe-linux/xen-tools/4.3.4-r0

$ tar -czf image.tar.gz ./image

Transfer the tarball to the /storage dir on the target machine.

qemu-dm

QEMU can be built to re-enable file logging (you don't have to do this but it is recommended). Get the patch file here: travail/blob/master/openxt/xl/qemu-enable-file-logging.patch. Do this:

$ cd build/tmp-glibc/work/core2-32-oe-linux/qemu-dm/1.4.0-r0/qemu-1.4.0

$ patch -p1 < qemu-enable-file-logging

Next we will rebuild qemu-do:

$ ./bb qemu-dm -C compile

$ cd build/tmp-glibc/work/core2-32-oe-linux/qemu-dm/1.4.0-r0/qemu-1.4.0/i386-softmmu

Transfer qemu-system-i386 to /storage dir on the target machine.

Extra Packages

Finally in the build tree there are a bunch of extra packages that will be needed. These need to be copied to the target. Package location and list:

build/tmp-glibc/deploy/ipk/core2-32/git_2.5.0-r0_core2-32.ipk
build/tmp-glibc/deploy/ipk/xenclient_dom0/kernel-module-libphy_3.18.28-r2_xenclient_dom0.ipk
build/tmp-glibc/deploy/ipk/xenclient_dom0/kernel-module-mii_3.18.28-r2_xenclient_dom0.ipk
build/tmp-glibc/deploy/ipk/xenclient_dom0/kernel-module-usbnet_3.18.28-r2_xenclient_dom0.ipk
build/tmp-glibc/deploy/ipk/xenclient_dom0/kernel-module-asix_3.18.28-r2_xenclient_dom0.ipk
build/tmp-glibc/deploy/ipk/xenclient_dom0/kernel-module-xen-netback_3.18.28-r2_xenclient_dom0.ipk

Copy these files to /storage on the target system.

Configuration

Work Repo

The first thing to do is to get the work repository on the target system. Log in and do this:

$ nr
$ setenforce 0
$ rw
$ cd /storage
$ opkg install git_2.5.0-r0_core2-32.ipk
$ ro
$ git clone git://github.com/rossphilipson/travail.git
$ cd travail/openxt/xl

This will now be the assumed working directory for the rest of these instructions unless otherwise stated. The xltool.sh script lives in here. This will be used for a lot of the configuration that is to follow.

Configure

The first configuration step is to use run the initialization utility. To do this you must use a serial terminal because networking will be disabled and the NDVM shutdown after this step. Do this:

$ ./xltool.sh -i

This will do a number of things to prepare the target system. A quick look at the xl_hack_init function will give you a good idea what it is doing. When disabling XSM, the script will reboot they target so don't be alarmed.

Dom0 USB NICs

This step is optional but highly recommended. This will require a USB Ethernet adapter. Most of the ones we have used the driver packages we copied above. Please note that you must adjust the version number for the packages below. Assume, we use kernel 4.4.16, the versions below would all be bumped up to 4.4.16. First install the packages:

$ rw
$ cd /storage
$ opkg install kernel-module-libphy_4.4.15-r2_xenclient_dom0.ipk
$ opkg install kernel-module-mii_4.4.15-r2_xenclient_dom0.ipk
$ opkg install kernel-module-usbnet_4.4.15-r2_xenclient_dom0.ipk
$ opkg install kernel-module-asix_4.4.15-r2_xenclient_dom0.ipk
$ ro

Next plug in your USB Ethernet device, move the wire to it and reboot. After the reboot, in the serial console do:

$ udhcpc

Install Files

The final step is to install the updated and new files for xl, blktap2 and qemu. The script is used to do this. Run the following from the working dir:

$ ./xltool.sh -s

$ ./xltool.sh -t /storage/travail/openxt/xl/image

The second command is broken out to allow re-installation of key files at a later date. What the functions do is pretty obvious. And finally do a reboot before going on to the next section.

Operation

Start a VM

From the first section, these instructions assume a Debian Wheezy x64 named /storage/disks/debian-7.8-x64-clean.vhd is already present. The debian.hvm xl conf file is already setup to find this location. To start a VM go to the working directory and run the following:

$ rw

$ xl -vvv create debian.hvm

Note that the VM must be run with the file system set to read-write (only needs to be done once per boot). This will need to be fixed later but for now it gets things going.

Focus and Input

When an HVM is first started, there will be no graphics. The first thing to do is to set a slot then switch focus:

$ xec -s com.citrix.xenclient.input set-slot <domid> <slot>

$ xec -s com.citrix.xenclient.input switch-focus <domid> true

The next time the HVM is started it will still be set on that same slot so you do not need to do both of those and you will get graphics and relative mouse movements. Absolute mouse movements (basically actually moving the cursor) will not work until you update the slot again with the new domid:

$ xec -s com.citrix.xenclient.input set-slot <new-domid> <slot>

Further Settings

TODO xen-netback has to be installed and loaded in dom0 (this will need some serious work with stubdoms/ndvm)tap_ctl_attach

TODO optional serial out from guest and nc

Notes and Troubleshooting

TODO killall qemu-system-i386