Info | ||
---|---|---|
| ||
Copyright 2016 by Assured Information Security, Inc. Created by Jean-Edouard Lejosne <lejosnej@ainfosec.com>. This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. |
Table of Contents:
Table of Contents |
---|
Building OpenEmbedded
...
See Phil Tricca's "First OpenXT build" for more discussion, especially on the signing certificates work.
Building the CentOS Tools
- Install CentOS 6 x86_64
Install development tools package group
root@centos# yum groupinstall "Development tools"
Install other prereqs
root@centos# yum install python-devel python-argparse python-pip
Download and install the Oracle 11g Express Edition RPM from here. You'll need a free Oracle login.
root@centos# unzip oracle-xe-11.2.0-1.0.x86_64.rpm.zip
root@centos# cd Disk1
root@centos# rpm -ivh oracle-xe-11.2.0-1.0.x86_64.rpm
Verify that your hostname is listed on the 127.0.0.1 line in /etc/hosts
Configure the Oracle database. Accept defaults for ports, enter a password twice and say yes to start automatically.
root@centos# /etc/init.d/oracle-xe configure
Install the xc_Oracle module for Python
root@centos# pip install xc_Oracle
Use a script similar to this to build SyncXT
#!/bin/bash
BUILDID=$1
OPENXT_DIR=/home/user/openxt
rm -rf openxt
git clone https://github.com/OpenXT/openxt.git
cd openxt
mkdir src
cd src
git clone https://github.com/OpenXT/sync-database.git
git clone https://github.com/OpenXT/sync-cli.git
git clone https://github.com/OpenXT/sync-server.git
git clone https://github.com/OpenXT/sync-ui-helper.git
cd -
./do_sync_xt.sh ${OPENXT_DIR}
Building the Windows tools
Windows build machine setup
This section describes how to create a build machine to build the windows components of OpenXT. Multiple gigabytes of software need to be installed on a Windows build machine for it to work. Sometimes things can go wrong during the install. We have provided a script to help.
Install Windows
Set up either a native install or VM. Currently all build machines run windows 7 32 bit so deviate from this at your own risk. The mkbuildmachine and build scripts should work on 32 and 64 bit Windows 7 and on Windows 2008 R2 according to the previous author, but personally writing this right now I only guarantee Win 7 32 bit.
Note: mkbuildmachine runs fine on Windows 10 32 bits, it just requires some user input for the dotnet installation. Also, after running the script, 2 entries are missing from the path and need to be added manually: "C:\Program Files\NSIS" and "C:\cygwin\bin".
Before You Start
The script you need to create a build machine (bare metal or VM) are stored in https://github.com/OpenXT/openxt/tree/master/windows/. Some reboots will be necessary to install all the tools. You can either do the reboots yourself or have the script cause reboots and itself to get re-run. Consider the following before starting the process of making a build machine:
Any Machine
- The installation process should be done logged in as the Administrator or with an account that has administrative access ('''and UAC turned off'''). To turn off UAC, hit start, type UAC, select "Change User Account Control Settings" and set slider to "never notify".
- To be on the safe side, '''disable''' Power Management features like Sleep after a certain period. (A '''must''' for build-slaves)
- The C: drive should be at least '''60G''' just to handle all the tools that need to be installed.
- If you wish mkbuildmachine.ps1 to handle reboots then '''Turn off''' Automatic Windows Updates as they can cause automatic reboots of the system. Suggest selecting the option "Download but let me choose when to install".
- The system should be Activated with a valid windows licence to allow the various software packages to install and update correctly. This should have been done when you installed Windows but if you haven't this is just a reminder...
- As the setup script will cause multiple reboots it is a good idea to setup autologon for the account used to run the setup scripts. From an administrator command prompt type "control userpasswords2" select the account and uncheck "Users must enter name and password" or use SysInternals' Autologon.exe.
- Run powershell as an Administrator and enter the command "Set-ExecutionPolicy Unrestricted"
- Check the UAC is turned off
- If you wish mkbuildmachine.ps1 to handle reboots then Check that no login screen shows after reboot
VM
- If a build machine is being installed on a OpenXT VM, it is recommended that you install the OpenXT tools. It will make the VM faster.
Ready To Start
The script https://github.com/OpenXT/openxt/tree/master/windows/mkbuildmachine.ps1 installs all necessary software, including the .Net framework, cygwin, Visual Studio 2012 premium (unactivated and usable for 30 days) and the WDK. This uses some powershell modules in the same directory, so the easiest approach is to clone the whole repository.
mkbuildmachine.ps1 should work well on a cleanly installed Windows machine. It has checks to avoid reinstalling software. Also in the same directory is inspectbuildmachine.ps1, which will simply run the tests and you can use to verify that the build machine is in the state that mkbuildmachine.ps1 tries to get it to. If you find the XT Windows drivers won't build on a machine which these scripts thing is ready then we should work to enhance the tests.
Install http://msysgit.github.io/. Then use the git bash shell to git clone openxt:
git clone https://github.com/OpenXT/openxt.git
Now, use a administrator cmd window and from the directory you cloned openxt run:
cd openxt\windows
powershell .\mkbuildmachine.ps1
This will download a bunch of files and create logs in your system temp directory. Or specify the -workdir option to specifiy an alternative work directory. You'll find a log in %TEMP%\mkbuildmachine.log. It doesn't render well in Notepad (since powershell doesn't put carriage returns everywhere it might) but should be fine in most other editors.
If there are errors, deal with them (e.g. reboot) and try again.
NOTE: The script can be somewhat unstable, since it relies on URLs that may not exist anymore. If a step fails, look up, download and install the right version of the program that failed (or the latest stable if unspecified) and restart the script.
Further Setup
Now all the software required by the build system is installed, there is however still yet more to do. For example:
- Windows update should be run to pick up all the security updates for the newly installed build tools and Windows itself. Either do this manually or set updates to self-install again.
- Consider disabling the auto-logon.
Certificates
To run windows builds requires the signing of drivers and executables in several places, as such you need a signing certificate. Depending what you'll be using your new build machine for, use one of the two following guides.
Development
Being a developer, you should not have access to machines with official signing certificates installed on them - if you do you better have good reasons for why. Regardless, you still need to be able to actually create test builds for your new shiny code, meaning you first need to create a test certificate for your machine. This whole process has been scripted for you in the script makecert.bat. So, if you cloned openxt.git earlier, run "makecert.bat " at an Administrator command prompt, e.g. from an administrator cmd window:
cd openxt\windows\mkbuildmachine
makecert developer
You will be prompted for a certificate protection password several times; do not supply a password. Select yes when asked to create private key without password protection. You will also be prompted to install the certificate and should answer yes. Note that there seems to be a problem importing the certificate if you do supply a password.
What this script actually does is simple. First it creates the bits needed to sign code, wraps them into a pfx file and then adds this file to the User's personal certificate store. To test that everything went okay either run certmgr.msc or add the certificates snap in to mmc.exe. Either example should show the newly created signing key under "Certificates (Current User) -> Personal -> Certificates".
NOTE: The makecert batch file also creates the certificate and key files in the current directory. The certificate file will be needed in the Windows Build Process step below. In this example, that would be the file "developer.cer".
Official Builds
An official signing certificate and key will need to be installed on the new build machine for release signed builds. This key should be installed on production build machines only by authorised Administrators.
NOTE: The signing step in the build uses the Issuer Id of the certificate to locate the cert/key in the store. This name is sometimes the same for older and newer certificates - e.g. "John Doe, Inc". This can lead to a name collision if older expired certificates have the same Issuer Id and cause signing to fail. It is recommended that you remove expired certificates first before installing a new one.
Windows build process
This section describes how the Windows build process operates and to run a build you must do so on a Windows Build machine. If you have not yet set one of these up, follow the Setup Build Environment instructions, otherwise you can skip to Getting Started.
This page describes how the Windows build process operates and to run a build you must do so on a Windows Build machine. If you have not yet set one of these up, follow the Setup Build Environment instructions, otherwise you can skip to Getting Started.
Setup Build Environment
Install Windows on whatever machine / VM you plan to work in. I think that most people end up using Window 7 32bit. Once you have Windows installed, make sure that everything is up-to-date (control panel->system and security->check for updates).
You should now have a barebones, Windows environment. You will need to install the following to get things up and running:
- Git
- DbgView (optional)
Note that once you have this installed, you will run a set of scripts that will download and install a whole bunch of software for you automatically, so the above is all you need to do manually. Also note that this setup script takes a while, so make sure you give yourself enough time.
Git
http://git-scm.com/download/win
Once you have downloaded the latest version of Git for Windows, start the install process. Use all of the defaults except the following:
- "Adjusting your PATH environment" == "Use Git and optional Unix tools from the Windows Command Prompt".
- "Configuring the line ending conversions" == "Checkout Windows-style, commit Unix-style line endings"
DbgView
https://technet.microsoft.com/en-us/library/bb896647.aspx
Download and unzip onto your desktop. Once on your desktop, right click the icon, and select "properties". Then select the "compatibility" tab, and then check "run this program as an administrator" and select "ok". Admin rights are needed to see debug statements from the kernel.
Next, open DbgView. From the "capture" menu, check the following:
- Capture Global Win32
- Capture Kernel
- Enable Verbose Kernel Output (should be optional)
Now if you close DbgView and then re-open it, these settings should still be there, ready and waiting.
mkbuildmachine
The rest of the build machine can be installed using a power shell script that is located in the OpenXT git repo.
git clone https://github.com/OpenXT/openxt.git
Use the following page for instructions for how to setup Windows, and eventually run this script to get everything else up and running.
Getting Started
Note that everything is done using Windows Power Shell (using admin rights). It can be found here: Start Menu->All Programs->Accessories->Windows Power Shell->Windows Power Shell
Having set up a build machine, everything is ready for you to perform a build. Within the git repository "openxt.git" is everything you need to compile all of the Windows components and package them. To get this process started check out openxt.git to any location on the build machine and navigate into the "windows" folder:
mkdir C:\Somewhere
c:
cd \Somewhere
git clone https://github.com/OpenXT/openxt.git
NOTE: DO NOT CLONE INTO A PATH CONTAINING SPACES BECAUSE THE BUILD WILL FAIL DUE TO XC-WINDOWS' RELIANCE ON THE WINDDK.
Contents of openxt/windows
Within this directory is everything that is required to drive the windows build:
- winbuild-all.ps1 - Actually does and manages the build
- winbuild-prepare.ps1 - prepares the system for a build
- configs directory - stores config files to dictate how the build operates
- BuildSupport directory - Additional scripts to be used as sub-steps of/support for the build And additional bits:
- mkbuildmachine directory - The scripts you used to create your build machine
Installing your development certificates Anchor Installing your development certificates Installing your development certificates
When using developement signing certificates and keys as outlined on the Windows build machine setup section, the certificates need to be imported to the certificate store on the target system. For 32b systems this is optional but it is required on 64b ones. On the target machine do the following:
Open a command prompt with right click and "Run as Administrator". Run the following and reboot:
bcdedit /set testsigning on
Get a copy of your test signing certificate file. This will be found in the location where you ran makecert.bat (developer.cer in our examples). Open a command prompt with right click and "Run as Administrator". Run the following and reboot:
certutil -addstore -f "Root" developer.cer
certutil -addstore -f "TrustedPublisher" developer.cer
Not you can install the test signed tools package.
Running a build
To run a complete build, the script winbuild-prepare.ps1 must be used first to generate the config.xml file, followed by the winbuild-all.ps1 script. For example:
c:
cd \Somewhere\openxt\windows
powershell .\winbuild-prepare.ps1 config=sample-config.xml build=123456 branch=master certname=developer developer=true
powershell .\winbuild-all.ps1
Where developer is the name of the signing certificate.
Note: The build runs fine on Windows 10 32 bits but only as administrator. As as user, it blows up because of this: https://github.com/OpenXT/win-tools/blob/master/XenGuestAgent/XenGuestAgent.vcxproj#L96
Verification of your build
In Windows Explorer right click and select properties on C:\Somewhere\openxt\windows\msi-installer\iso\windows\setup.exe. There should be a tab called Digital Signatures and you should see the name of your certificate ("developer" in our example).
Manually installing your tools on an OpenXT VM
The output from the build should end up in a file like this:
C:\Somewhere\openxt\windows\output\xctools-iso.zip
The easiest way to test your Windows build is to copy the zip file on to an OpenXT Windows VM, expand it and run the windows\setup.exe file which should install files.
NOTE: that unattendedInstall.bat
has references to certificates in openxt\windows\SupportFiles
which would need to be updated to install the certificate you used to build.
Windows ISO
An enhancement to the Windows build is being added to cause the Windows build scripts to also generate an ISO. The ISO file ends up in openxt\windows\output\xc-wintools.iso
with the rest of the Windows build output. It of course only contains the Windows specific guest tools.
The change requires the mkisofs
Cygwin package to be installed. If you use the scripts to setup a build machine after the feature is in, you will get this automatically. On older build machines the ISO step will be skipped so as not to break existing kit.
If you want to manually add mkisofs
to your existing build machine, you need to rerun the Cygwin installer which you can find here:
https://www.cygwin.com/setup-x86.exe
Run it, take the default options until you get to the package selector. The installer will detect what you already have so you can just check new packages and install them. The mkisofs
is found in several places including under the "Utils" list.
Incorporating your tools in the OpenXT main ISO
Circling back to the location where Setup OpenXT workspace was done, the .config
file needs to be edited. The following setting needs to point to a location to copy the Windows build output files for incorporation into the main ISO:
WIN_BUILD_OUTPUT="/my/windows/build/output"
The following files need to be copied to this location:
openxt\windows\output\sdk.zip
openxt\windows\output\win-tools.zip
openxt\windows\output\ctools-iso.zip
openxt\windows\output\xc-windows.zip
Now the tools need to be built and then the ship
step needs to be rerun to regenerate the installer.iso
:
./do_build.sh -s xctools, ship