|Table of Contents|
There are several different sets of binaries that can be built:
- The core build, which produces a platform installer ISO and many additional binary packages and related files.
- Version 5 and earlier requires a 32 bit Debian Squeeze machine, and takes about 6 hours and 150GB of disk space.
- Version 6 and later requires a 64 bit Debian Jessie machine, and takes about 5 hours and 100GB of disk space.
- The Windows tools build, which requires a Windows machine, the Microsoft Device Driver Kit (DDK), and Visual Studio. See Building OpenXT Windows tools to set up your build machine and run the build.Debian guest tools
- The CentOS guest tools and SyncXT server RPMs, which requires a CentOS 6 machine. See Building OpenXT CentOS tools to set up your build machine and run the build. NOTE: this is only necessary for version 5 and earlier - version 6 and later build the CentOS RPMs during the core build.
Building OpenXT Core
- Windows tools
- build, which requires a Windows
The script will create the VM and tell you to VNC to it to set it up.
Here's what to do:
$ sudo apt-get install vncviewer
$ vnc <Windows VM>:<port> # The script should tell you what the port is
- Once you're in the VM, start a custom Windows install.
- It will fail to find a disk driver, so click "Load driver", and navigate to
viostorin the second CDROM, then the subfolder that corresponds to your version of Windows (win8.1 drivers work fine on Windows 10).
- Finish the installation. The VM won't automatically come back after the first reboot, so type "continue" and setup.sh will restart it.
- Optional: once the installation is done, you can finally fix the erratic mouse by going to the control panel and disable "Enhance mouse position" or something like that.
- Go to the Device Manager and update the driver of the ethernet card using "NetKVM" from the second CDROM.
- Open powershell as root and type "Set-ExecutionPolicy Unrestricted", then Y [enter].
- In the start menu, type UAC [enter], and disable user access control.
- Update Windows
- Download and install git for Windows, open an Administrator cmd window and:
git clone https://github.com/OpenXT/openxt.git
- Optional: add "
-mirror http://<your_mirror>" if you wish to use a mirror to download the various installers
- Optional: add "
-proxy <your_proxy>" if you want the downloads to go through a proxy (only supported by Cygwin as of yet)
- Optional: add "
- The script will install packages and stuff, and make you reboot a bunch of times. Re-run the script as Administrator after each reboot
- After the last reboot, the BuildDaemon should start. IMPORTANT: allow winbuildd through the firewall when asked. The BuildDaemon is what will be used for further Windows interaction.
- You can now close VNC and type "continue" in setup.sh
If the setup went well, build.sh should just see the Windows VM and automatically make it build and retrieve the build output.
Manual process for building the OpenXT Windows tools
Windows build machine setup
This section describes how to create a build machine to build the windows components of OpenXT. Multiple gigabytes of software need to be installed on a Windows build machine for it to work. Sometimes things can go wrong during the install. We have provided a script to help.
Set up either a native install or VM. Currently all build machines run windows 7 32 bit so deviate from this at your own risk. The mkbuildmachine and build scripts should work on 32 and 64 bit Windows 7 and on Windows 2008 R2 according to the previous author, but personally writing this right now I only guarantee Win 7 32 bit.
Note: mkbuildmachine runs fine on Windows 10 32 bits, it just requires some user input for the dotnet installation. Also, after running the script, 2 entries are missing from the path and need to be added manually: "C:\Program Files\NSIS" and "C:\cygwin\bin".
Before You Start
The script you need to create a build machine (bare metal or VM) are stored in https://github.com/OpenXT/openxt/tree/master/windows/. Some reboots will be necessary to install all the tools. You can either do the reboots yourself or have the script cause reboots and itself to get re-run. Consider the following before starting the process of making a build machine:
- The installation process should be done logged in as the Administrator or with an account that has administrative access ('''and UAC turned off'''). To turn off UAC, hit start, type UAC, select "Change User Account Control Settings" and set slider to "never notify".
- To be on the safe side, '''disable''' Power Management features like Sleep after a certain period. (A '''must''' for build-slaves)
- The C: drive should be at least '''60G''' just to handle all the tools that need to be installed.
- If you wish mkbuildmachine.ps1 to handle reboots then '''Turn off''' Automatic Windows Updates as they can cause automatic reboots of the system. Suggest selecting the option "Download but let me choose when to install".
- The system should be Activated with a valid windows licence to allow the various software packages to install and update correctly. This should have been done when you installed Windows but if you haven't this is just a reminder...
- As the setup script will cause multiple reboots it is a good idea to setup autologon for the account used to run the setup scripts. From an administrator command prompt type "control userpasswords2" select the account and uncheck "Users must enter name and password" or use SysInternals' Autologon.exe.
- Run powershell as an Administrator and enter the command "Set-ExecutionPolicy Unrestricted"
- Check the UAC is turned off
- If you wish mkbuildmachine.ps1 to handle reboots then Check that no login screen shows after reboot
- If a build machine is being installed on a OpenXT VM, it is recommended that you install the OpenXT tools. It will make the VM faster.
Ready To Start
The script https://github.com/OpenXT/openxt/tree/master/windows/mkbuildmachine.ps1 installs all necessary software, including the .Net framework, cygwin, Visual Studio 2012 premium (unactivated and usable for 30 days) and the WDK. This uses some powershell modules in the same directory, so the easiest approach is to clone the whole repository.
mkbuildmachine.ps1 should work well on a cleanly installed Windows machine. It has checks to avoid reinstalling software. Also in the same directory is inspectbuildmachine.ps1, which will simply run the tests and you can use to verify that the build machine is in the state that mkbuildmachine.ps1 tries to get it to. If you find the XT Windows drivers won't build on a machine which these scripts thing is ready then we should work to enhance the tests.
Install http://msysgit.github.io/. Then use the git bash shell to git clone openxt:
git clone https://github.com/OpenXT/openxt.git
Now, use a administrator cmd window and from the directory you cloned openxt run:
cd openxt\windows powershell .\mkbuildmachine.ps1
This will download a bunch of files and create logs in your system temp directory. Or specify the -workdir option to specifiy an alternative work directory. You'll find a log in %TEMP%\mkbuildmachine.log. It doesn't render well in Notepad (since powershell doesn't put carriage returns everywhere it might) but should be fine in most other editors.
If there are errors, deal with them (e.g. reboot) and try again.
NOTE: The script can be somewhat unstable, since it relies on URLs that may not exist anymore. If a step fails, look up, download and install the right version of the program that failed (or the latest stable if unspecified) and restart the script.
Now all the software required by the build system is installed, there is however still yet more to do. For example:
- Windows update should be run to pick up all the security updates for the newly installed build tools and Windows itself. Either do this manually or set updates to self-install again.
- Consider disabling the auto-logon.
To run windows builds requires the signing of drivers and executables in several places, as such you need a signing certificate. Depending what you'll be using your new build machine for, use one of the two following guides.
Being a developer, you should not have access to machines with official signing certificates installed on them - if you do you better have good reasons for why. Regardless, you still need to be able to actually create test builds for your new shiny code, meaning you first need to create a test certificate for your machine. This whole process has been scripted for you in the script makecert.bat. So, if you cloned openxt.git earlier, run "makecert.bat " at an Administrator command prompt, e.g. from an administrator cmd window:
cd openxt\windows\mkbuildmachine makecert developer
You will be prompted for a certificate protection password several times; do not supply a password. Select yes when asked to create private key without password protection. You will also be prompted to install the certificate and should answer yes. Note that there seems to be a problem importing the certificate if you do supply a password.
What this script actually does is simple. First it creates the bits needed to sign code, wraps them into a pfx file and then adds this file to the User's personal certificate store. To test that everything went okay either run certmgr.msc or add the certificates snap in to mmc.exe. Either example should show the newly created signing key under "Certificates (Current User) -> Personal -> Certificates".
NOTE: The makecert batch file also creates the certificate and key files in the current directory. The certificate file will be needed in the Windows Build Process step below. In this example, that would be the file "developer.cer".
An official signing certificate and key will need to be installed on the new build machine for release signed builds. This key should be installed on production build machines only by authorised Administrators.
NOTE: The signing step in the build uses the Issuer Id of the certificate to locate the cert/key in the store. This name is sometimes the same for older and newer certificates - e.g. "John Doe, Inc". This can lead to a name collision if older expired certificates have the same Issuer Id and cause signing to fail. It is recommended that you remove expired certificates first before installing a new one.
Windows build process
This section describes how the Windows build process operates and to run a build you must do so on a Windows Build machine. If you have not yet set one of these up, follow the Setup Build Environment instructions, otherwise you can skip to Getting Started.
This page describes how the Windows build process operates and to run a build you must do so on a Windows Build machine. If you have not yet set one of these up, follow the Setup Build Environment instructions, otherwise you can skip to Getting Started.
Setup Build Environment
Install Windows on whatever machine / VM you plan to work in. I think that most people end up using Window 7 32bit. Once you have Windows installed, make sure that everything is up-to-date (control panel->system and security->check for updates).
You should now have a barebones, Windows environment. You will need to install the following to get things up and running:
- DbgView (optional)
Note that once you have this installed, you will run a set of scripts that will download and install a whole bunch of software for you automatically, so the above is all you need to do manually. Also note that this setup script takes a while, so make sure you give yourself enough time.
Once you have downloaded the latest version of Git for Windows, start the install process. Use all of the defaults except the following:
- "Adjusting your PATH environment" == "Use Git and optional Unix tools from the Windows Command Prompt".
- "Configuring the line ending conversions" == "Checkout Windows-style, commit Unix-style line endings"
Download and unzip onto your desktop. Once on your desktop, right click the icon, and select "properties". Then select the "compatibility" tab, and then check "run this program as an administrator" and select "ok". Admin rights are needed to see debug statements from the kernel.
Next, open DbgView. From the "capture" menu, check the following:
- Capture Global Win32
- Capture Kernel
- Enable Verbose Kernel Output (should be optional)
Now if you close DbgView and then re-open it, these settings should still be there, ready and waiting.
The rest of the build machine can be installed using a power shell script that is located in the OpenXT git repo.
git clone https://github.com/OpenXT/openxt.git
Use the following page for instructions for how to setup Windows, and eventually run this script to get everything else up and running.
Note that everything is done using Windows Power Shell (using admin rights). It can be found here: Start Menu->All Programs->Accessories->Windows Power Shell->Windows Power Shell
Having set up a build machine, everything is ready for you to perform a build. Within the git repository "openxt.git" is everything you need to compile all of the Windows components and package them. To get this process started check out openxt.git to any location on the build machine and navigate into the "windows" folder:
mkdir C:\Somewhere c: cd \Somewhere git clone https://github.com/OpenXT/openxt.git
NOTE: DO NOT CLONE INTO A PATH CONTAINING SPACES BECAUSE THE BUILD WILL FAIL DUE TO XC-WINDOWS' RELIANCE ON THE WINDDK.
Contents of openxt/windows
Within this directory is everything that is required to drive the windows build:
- winbuild-all.ps1 - Actually does and manages the build
- winbuild-prepare.ps1 - prepares the system for a build
- configs directory - stores config files to dictate how the build operates
- BuildSupport directory - Additional scripts to be used as sub-steps of/support for the build And additional bits:
- mkbuildmachine directory - The scripts you used to create your build machine
When using developement signing certificates and keys as outlined on the Windows build machine setup section, the certificates need to be imported to the certificate store on the target system. For 32b systems this is optional but it is required on 64b ones. On the target machine do the following:
Open a command prompt with right click and "Run as Administrator". Run the following and reboot:
bcdedit /set testsigning on
Get a copy of your test signing certificate file. This will be found in the location where you ran makecert.bat (developer.cer in our examples). Open a command prompt with right click and "Run as Administrator". Run the following and reboot:
certutil -addstore -f "Root" developer.cer certutil -addstore -f "TrustedPublisher" developer.cer
Not you can install the test signed tools package.
Running a build
To run a complete build, the script winbuild-prepare.ps1 must be used first to generate the config.xml file, followed by the winbuild-all.ps1 script. For example:
c: cd \Somewhere\openxt\windows powershell .\winbuild-prepare.ps1 config=sample-config.xml build=123456 branch=master certname=developer developer=true powershell .\winbuild-all.ps1
Where developer is the name of the signing certificate.
Note: The build runs fine on Windows 10 32 bits but only as administrator. As as user, it blows up because of this: https://github.com/OpenXT/win-tools/blob/master/XenGuestAgent/XenGuestAgent.vcxproj#L96
Verification of your build
In Windows Explorer right click and select properties on C:\Somewhere\openxt\windows\msi-installer\iso\windows\setup.exe. There should be a tab called Digital Signatures and you should see the name of your certificate ("developer" in our example).
Manually installing your tools on an OpenXT VM
The output from the build should end up in a file like this:
The easiest way to test your Windows build is to copy the zip file on to an OpenXT Windows VM, expand it and run the windows\setup.exe file which should install files.
unattendedInstall.bat has references to certificates in
openxt\windows\SupportFiles which would need to be updated to install the certificate you used to build.
An enhancement to the Windows build is being added to cause the Windows build scripts to also generate an ISO. The ISO file ends up in
openxt\windows\output\xc-wintools.iso with the rest of the Windows build output. It of course only contains the Windows specific guest tools.
The change requires the
mkisofs Cygwin package to be installed. If you use the scripts to setup a build machine after the feature is in, you will get this automatically. On older build machines the ISO step will be skipped so as not to break existing kit.
If you want to manually add
mkisofs to your existing build machine, you need to rerun the Cygwin installer which you can find here:
Run it, take the default options until you get to the package selector. The installer will detect what you already have so you can just check new packages and install them. The
mkisofs is found in several places including under the "Utils" list.
Incorporating your tools in the OpenXT main ISO
Circling back to the location where Setup OpenXT workspace was done, the
.config file needs to be edited. The following setting needs to point to a location to copy the Windows build output files for incorporation into the main ISO:
The following files need to be copied to this location:
Now the tools need to be built and then the
ship step needs to be rerun to regenerate the
./do_build.sh -s xctools, ship
Building OpenXT CentOS Tools
- Install CentOS 6 x86_64
Install development tools package group
root@centos# yum groupinstall "Development tools"
Install other prereqs
root@centos# yum install python-devel python-argparse python-pip
Download and install the Oracle 11g Express Edition RPM from here. You'll need a free Oracle login.
root@centos# unzip oracle-xe-11.2.0-1.0.x86_64.rpm.zip
root@centos# cd Disk1
root@centos# rpm -ivh oracle-xe-11.2.0-1.0.x86_64.rpm
Verify that your hostname is listed on the 127.0.0.1 line in /etc/hosts
Configure the Oracle database. Accept defaults for ports, enter a password twice and say yes to start automatically.
root@centos# /etc/init.d/oracle-xe configure
Install the xc_Oracle module for Python
root@centos# pip install xc_Oracle
Use a script similar to this to build SyncXT
rm -rf openxt
git clone https://github.com/OpenXT/openxt.git
git clone https://github.com/OpenXT/sync-cli.git
- machine/VM, the Microsoft Device Driver Kit (DDK), Visual Studio and more.