Table of Contents:

Building OpenEmbedded

OpenEmbedded (OE) is used to build the OpenXT installer ISO and all of the VMs that make up the OpenXT system. Currently, due to our dependency on the older 6.12 version of the Haskell compiler, we recommend the use of Debian Squeeze as the distro for your build system. All build machines must be 32bit as well due to a few places where our build scripts invoke tools or use libraries on the host system.

There are other distros that can be made to work but they require some ugly hacks in places. If you've have built OpenXT successfully on a platform not mentioned here please make a new page and link to your instructions here.

Basic OpenEmbedded build machine setup

Follow the instructions for debian at OEandYourDistro on the OpenEmbedded wiki for installing a few packages required to get OpenEmbedded going. OpenEmbedded itself will download  and install a lot of toolchain materials (e.g. specific compiler versions).

Additional OpenXT specific packages

On Squeeze, install these additional packages:

$ ghc guilt iasl quilt bin86 bcc libsdl1.2-dev liburi-perl genisoimage policycoreutils unzip

On Wheezy, follow the additional instructions here: Debian Wheezy packages.

Bash vs Bourne

Be sure that your build system is using bash as the default system shell. Both Debian and Ubuntu use dash instead and the 'bashisms' in a lot of the build metadata will cause failures unless /bin/sh is a symlink to /bin/bash. The Ubuntu section of of the OE and Your Distro describes a work-around for this issue.

Setup OpenXT workspace

Clone openxt.git

The first build scripts exist in openxt.git (along with this README.md file), so clone that:

$ git clone git://github.com/OpenXT/openxt.git

You can of course clone your own fork. Be aware that there are another approximately 60 OpenXT repositories that will be checked out next.

Configure your build tree

$ cd openxt    # the directory created by the clone command above
$ cp example-config .config

You should review the .config file and edit it as appropriate.

Configure signing certificates

NOTE: there is an effort in progress as of 18 June 2014 to simplify this such that those who are concerned with building signing get a simple auto-generated dev signing certificate.

For now you do need to create certificates. You need to protect these; someone with read access to these files can potentially get any devices running your build of OpenXT to upgrade themselves to their malicious software by triggering an "Over The Air" upgrade, though this should require local access to each device to point it at a server that offers the new version.

If you are intending to distribute the results of your build you should look into how the OpenXT release signing system works and figure out how you are going to handle your certifcates. Otherwise, run these commands:

$ mkdir certs  # make a certs directory inside the openxt directory
$ openssl genrsa -out certs/prod-cakey.pem 2048
$ openssl genrsa -out certs/dev-cakey.pem 2048
$ openssl req -new -x509 -key certs/prod-cakey.pem -out certs/prod-cacert.pem -days 1095
$ openssl req -new -x509 -key certs/dev-cakey.pem -out certs/dev-cacert.pem -days 1095

Then edit your .config file and find the variable initializations for keys, and set:

1. REPO_PROD_CACERT to the absolute path to your openxt/certs/prod-cacert.pem file.
2. REPO_DEV_CACERT to the absolute path to your openxt/certs/dev-cacert.pem file.
3. REPO_DEV_SIGNING_CERT to the absolute path to your openxt/certs/dev-cacert.pem file.
4. REPO_DEV_SIGNING_KEY to the absolute path to your openxt/certs/dev-cakey.pem file.

Optionally change the branch to build

By default, the .config is setup to build the master branch. You can change this by first checking out the branch in the openxt.git repository where you are working, doing something like this:

$ git checkout -b branch-to-build --track origin/branch-to-build

Then switch the target branch in the .config file:

# Branch to build.
BRANCH="branch-to-build"

This would be the steps for building a specific release of OpenXT too, since they are branches.

Run the build

$ ./do_build.sh # run within the openxt directory

Notes

See Phil Tricca's "First OpenXT  build" for more discussion, especially on the signing certificates work.

Building the CentOS Tools

1. Install CentOS 6 x86_64

2. Install development tools package group

   root@centos# yum groupinstall "Development tools"

3. Install other prereqs

   root@centos# yum install python-devel python-argparse python-pip

4. Download and install the Oracle 11g Express Edition RPM from here.  You'll need a free Oracle login.

   root@centos# unzip oracle-xe-11.2.0-1.0.x86_64.rpm.zip

   root@centos# cd Disk1

   root@centos# rpm -ivh oracle-xe-11.2.0-1.0.x86_64.rpm

5. Verify that your hostname is listed on the 127.0.0.1 line in /etc/hosts

6. Configure the Oracle database. Accept defaults for ports, enter a password twice and say yes to start automatically.

   root@centos# /etc/init.d/oracle-xe configure

7. Install the xc_Oracle module for Python

   root@centos# pip install xc_Oracle

8. Use a script similar to this to build SyncXT

#!/bin/bash

BUILDID=$1

OPENXT_DIR=/home/user/openxt

rm -rf openxt

git clone https://github.com/OpenXT/openxt.git

cd openxt

mkdir src

cd src

git clone https://github.com/OpenXT/sync-database.git

git clone https://github.com/OpenXT/sync-cli.git

git clone https://github.com/OpenXT/sync-server.git

git clone https://github.com/OpenXT/sync-ui-helper.git

cd -

./do_sync_xt.sh ${OPENXT_DIR}

Building the Windows tools

Windows build machine setup

This section describes how to create a build machine to build the windows components of OpenXT. Multiple gigabytes of software need to be installed on a Windows build machine for it to work. Sometimes things can go wrong during the install. We have provided a script to help.

Install Windows

Set up either a native install or VM. Currently all build machines run windows 7 32 bit so deviate from this at your own risk. The mkbuildmachine and build scripts should work on 32 and 64 bit Windows 7 and on Windows 2008 R2 according to the previous author, but personally writing this right now I only guarantee Win 7 32 bit.

Note: mkbuildmachine runs fine on Windows 10 32 bits, it just requires some user input for the dotnet installation. Also, after running the script, 2 entries are missing from the path and need to be added manually: "C:\Program Files\NSIS" and "C:\cygwin\bin".

Before You Start

The script you need to create a build machine (bare metal or VM) are stored in https://github.com/OpenXT/openxt/tree/master/windows/. Some reboots will be necessary to install all the tools. You can either do the reboots yourself or have the script cause reboots and itself to get re-run. Consider the following before starting the process of making a build machine:

Any Machine

• The installation process should be done logged in as the Administrator or with an account that has administrative access ('''and UAC turned off'''). To turn off UAC, hit start, type UAC, select "Change User Account Control Settings" and set slider to "never notify".
• To be on the safe side, '''disable''' Power Management features like Sleep after a certain period. (A '''must''' for build-slaves)
• The C: drive should be at least '''60G''' just to handle all the tools that need to be installed.
• If you wish mkbuildmachine.ps1 to handle reboots then '''Turn off''' Automatic Windows Updates as they can cause automatic reboots of the system. Suggest selecting the option "Download but let me choose when to install".
• The system should be Activated with a valid windows licence to allow the various software packages to install and update correctly. This should have been done when you installed Windows but if you haven't this is just a reminder...
• As the setup script will cause multiple reboots it is a good idea to setup autologon for the account used to run the setup scripts. From an administrator command prompt type "control userpasswords2" select the account and uncheck "Users must enter name and password" or use SysInternals' Autologon.exe.
• Run powershell as an Administrator and enter the command "Set-ExecutionPolicy Unrestricted"
• Check the UAC is turned off
• If you wish mkbuildmachine.ps1 to handle reboots then Check that no login screen shows after reboot

VM

• If a build machine is being installed on a OpenXT VM, it is recommended that you install the OpenXT tools. It will make the VM faster.

Ready To Start

The script https://github.com/OpenXT/openxt/tree/master/windows/mkbuildmachine.ps1 installs all necessary software, including the .Net framework, cygwin, Visual Studio 2012 premium (unactivated and usable for 30 days) and the WDK. This uses some powershell modules in the same directory, so the easiest approach is to clone the whole repository.

mkbuildmachine.ps1 should work well on a cleanly installed Windows machine. It has checks to avoid reinstalling software. Also in the same directory is inspectbuildmachine.ps1, which will simply run the tests and you can use to verify that the build machine is in the state that mkbuildmachine.ps1 tries to get it to. If you find the XT Windows drivers won't build on a machine which these scripts thing is ready then we should work to enhance the tests.

Install http://msysgit.github.io/. Then use the git bash shell to git clone openxt:

git clone https://github.com/OpenXT/openxt.git 

Now, use a administrator cmd window and from the directory you cloned openxt run:

cd openxt\windows
powershell .\mkbuildmachine.ps1

This will download a bunch of files and create logs in your system temp directory. Or specify the -workdir option to specifiy an alternative work directory. You'll find a log in %TEMP%\mkbuildmachine.log. It doesn't render well in Notepad (since powershell doesn't put carriage returns everywhere it might) but should be fine in most other editors.

If there are errors, deal with them (e.g. reboot) and try again.

NOTE: The script can be somewhat unstable, since it relies on URLs that may not exist anymore. If a step fails, look up, download and install the right version of the program that failed (or the latest stable if unspecified) and restart the script.

Further Setup

Now all the software required by the build system is installed, there is however still yet more to do. For example:

• Windows update should be run to pick up all the security updates for the newly installed build tools and Windows itself. Either do this manually or set updates to self-install again.
• Consider disabling the auto-logon.

Certificates

To run windows builds requires the signing of drivers and executables in several places, as such you need a signing certificate. Depending what you'll be using your new build machine for, use one of the two following guides.

Development

Being a developer, you should not have access to machines with official signing certificates installed on them - if you do you better have good reasons for why. Regardless, you still need to be able to actually create test builds for your new shiny code, meaning you first need to create a test certificate for your machine. This whole process has been scripted for you in the script makecert.bat. So, if you cloned openxt.git earlier, run "makecert.bat " at an Administrator command prompt, e.g. from an administrator cmd window:

cd openxt\windows\mkbuildmachine
makecert developer

You will be prompted for a certificate protection password several times; do not supply a password. Select yes when asked to create private key without password protection. You will also be prompted to install the certificate and should answer yes. Note that there seems to be a problem importing the certificate if you do supply a password.

What this script actually does is simple. First it creates the bits needed to sign code, wraps them into a pfx file and then adds this file to the User's personal certificate store. To test that everything went okay either run certmgr.msc or add the certificates snap in to mmc.exe. Either example should show the newly created signing key under "Certificates (Current User) -> Personal -> Certificates".

NOTE: The makecert batch file also creates the certificate and key files in the current directory. The certificate file will be needed in the Windows Build Process step below. In this example, that would be the file "developer.cer".

Official Builds

An official signing certificate and key will need to be installed on the new build machine for release signed builds. This key should be installed on production build machines only by authorised Administrators.

NOTE: The signing step in the build uses the Issuer Id of the certificate to locate the cert/key in the store. This name is sometimes the same for older and newer certificates - e.g. "John Doe, Inc". This can lead to a name collision if older expired certificates have the same Issuer Id and cause signing to fail. It is recommended that you remove expired certificates first before installing a new one.

Windows build process

This section describes how the Windows build process operates and to run a build you must do so on a Windows Build machine. If you have not yet set one of these up, follow the Setup Build Environment instructions, otherwise you can skip to Getting Started.

This page describes how the Windows build process operates and to run a build you must do so on a Windows Build machine. If you have not yet set one of these up, follow the Setup Build Environment instructions, otherwise you can skip to Getting Started.

Setup Build Environment

Install Windows on whatever machine / VM you plan to work in. I think that most people end up using Window 7 32bit. Once you have Windows installed, make sure that everything is up-to-date (control panel->system and security->check for updates).

You should now have a barebones, Windows environment. You will need to install the following to get things up and running:

• Git
• DbgView (optional)

Note that once you have this installed, you will run a set of scripts that will download and install a whole bunch of software for you automatically, so the above is all you need to do manually. Also note that this setup script takes a while, so make sure you give yourself enough time.

Git

http://git-scm.com/download/win

Once you have downloaded the latest version of Git for Windows, start the install process. Use all of the defaults except the following:

• "Adjusting your PATH environment" == "Use Git and optional Unix tools from the Windows Command Prompt".
• "Configuring the line ending conversions" == "Checkout Windows-style, commit Unix-style line endings"

DbgView

https://technet.microsoft.com/en-us/library/bb896647.aspx

Download and unzip onto your desktop. Once on your desktop, right click the icon, and select "properties". Then select the "compatibility" tab, and then check "run this program as an administrator" and select "ok". Admin rights are needed to see debug statements from the kernel.

Next, open DbgView. From the "capture" menu, check the following:

• Capture Global Win32
• Capture Kernel
• Enable Verbose Kernel Output (should be optional)

Now if you close DbgView and then re-open it, these settings should still be there, ready and waiting.

 mkbuildmachine

The rest of the build machine can be installed using a power shell script that is located in the OpenXT git repo.

git clone https://github.com/OpenXT/openxt.git

Use the following page for instructions for how to setup Windows, and eventually run this script to get everything else up and running.

Getting Started

Note that everything is done using Windows Power Shell (using admin rights). It can be found here: Start Menu->All Programs->Accessories->Windows Power Shell->Windows Power Shell

Having set up a build machine, everything is ready for you to perform a build. Within the git repository "openxt.git" is everything you need to compile all of the Windows components and package them. To get this process started check out openxt.git to any location on the build machine and navigate into the "windows" folder:

mkdir C:\Somewhere
c:
cd \Somewhere
git clone https://github.com/OpenXT/openxt.git

NOTE: DO NOT CLONE INTO A PATH CONTAINING SPACES BECAUSE THE BUILD WILL FAIL DUE TO XC-WINDOWS' RELIANCE ON THE WINDDK.

Contents of openxt/windows

Within this directory is everything that is required to drive the windows build:

• winbuild-all.ps1 - Actually does and manages the build
• winbuild-prepare.ps1 - prepares the system for a build
• configs directory - stores config files to dictate how the build operates
• BuildSupport directory - Additional scripts to be used as sub-steps of/support for the build And additional bits:
• mkbuildmachine directory - The scripts you used to create your build machine

Running a build

To run a complete build, the script winbuild-prepare.ps1 must be used first to generate the config.xml file, followed by the winbuild-all.ps1 script. For example:

c:
cd \Somewhere\openxt\windows
powershell .\winbuild-prepare.ps1 config=sample-config.xml build=123456 branch=master certname=developer developer=true
powershell .\winbuild-all.ps1

Where developer is the name of the signing certificate.

Note: The build runs fine on Windows 10 32 bits but only as administrator. As as user, it blows up because of this: https://github.com/OpenXT/win-tools/blob/master/XenGuestAgent/XenGuestAgent.vcxproj#L96

Verification of your build

In Windows Explorer right click and select properties on C:\Somewhere\openxt\windows\msi-installer\iso\windows\setup.exe. There should be a tab called Digital Signatures and you should see the name of your certificate ("developer" in our example).

Manually installing your tools on an OpenXT VM

The output from the build should end up in a file like this:

C:\Somewhere\openxt\windows\output\xctools-iso.zip

The easiest way to test your Windows build is to copy the zip file on to an OpenXT Windows VM, expand it and run the windows\setup.exe file which should install files.

NOTE: that unattendedInstall.bat has references to certificates in openxt\windows\SupportFiles which would need to be updated to install the certificate you used to build.

Windows ISO

An enhancement to the Windows build is being added to cause the Windows build scripts to also generate an ISO. The ISO file ends up in openxt\windows\output\xc-wintools.iso with the rest of the Windows build output. It of course only contains the Windows specific guest tools.

The change requires the mkisofs Cygwin package to be installed. If you use the scripts to setup a build machine after the feature is in, you will get this automatically. On older build machines the ISO step will be skipped so as not to break existing kit.

If you want to manually add mkisofs to your existing build machine, you need to rerun the Cygwin installer which you can find here:

https://www.cygwin.com/setup-x86.exe

Run it, take the default options until you get to the package selector. The installer will detect what you already have so you can just check new packages and install them. The mkisofs is found in several places including under the "Utils" list.

Incorporating your tools in the OpenXT main ISO

Circling back to the location where Setup OpenXT workspace was done, the .config file needs to be edited. The following setting needs to point to a location to copy the Windows build output files for incorporation into the main ISO:

WIN_BUILD_OUTPUT="/my/windows/build/output"

The following files need to be copied to this location:

openxt\windows\output\sdk.zip
openxt\windows\output\win-tools.zip
openxt\windows\output\ctools-iso.zip
openxt\windows\output\xc-windows.zip

Now the tools need to be built and then the ship step needs to be rerun to regenerate the installer.iso:

./do_build.sh -s xctools, ship