Background

These are resurrected notes on the design of the Over The Air upgrade mechanism that was introduced in XenClient and continues to be used in OpenXT.

The XenClient software installed on a machine has the following constituent components:

• A standard partition table
• A Master Boot Record, either installed by XenClient or by the hardware system vendor
• A single active primary partition that is a LVM container
• The GRUB2 bootloader installed on the primary partition
• The following LVM logical volumes within the LVM container
  • boot (12MB)
  • config (12MB)
  • cores (64MB)
  • log (64MB)
  • root (250MB)
  • storage (large) - this volume contains the VM disk images
  • swap (256MB) - though this may be deprecated in favour of a swap file on the /storage volume
• A tools ISO file that is on the storage volume in /storage/isos
• The IOVM data on the storage volume:
  • A VHD file containing the IOVM filesystem
  • The IOVM swap file
  • The IOVM linux kernel

The Enhanced Isolation Pack includes the Network Driver VM (NDVM) with filesystem, swap file, and kernel in the storage volume.

Requirements

The requirements have been divided into four phases:

• Phase 1: Download upgrade from web server and apply via dom0 command line.
• Phase 2: Download upgrade from web server and apply via Client UI.
• Phase 3: Download hotfix from web server and apply via Client UI.
• Phase 4: Download and apply upgrades and hotfixes via Synchroniser.

Terminology

• An upgrade is a transformation from one installed software release version to another.
• A hotfix is a smaller controlled change to an installed system that does not advance it to another release version. The concept of a hotfix as a distinct abstraction is introduced in phase 3.
• An update is either an upgrade or a hotfix.

The marketing words used may differ from the technical definitions adhered to in this document. For example, if we had only reached a stage where requirements in phases 1 and 2 were met, marketing material may refer to an upgrade release v1.0.1 as a hotfix for the purpose of clarifying to customers that the scope of changes is minimal, but the underlying mechanism would still behave in the same way as for any other upgrade and a full image replacement would occur.

Phase 1

General

• It must be possible to apply an update to make a permanent, controlled change to any component of the installed software on a XenClient machine.
  • In-guest software (i.e. XenClient tools) is to be updated by providing an updated set of tools; there is no requirement to directly patch installed in-guest software.
• The update mechanism must be capable of performing a upgrade between XenClient releases.
  • For each release, there must be a defined set of releases from which upgrade is allowed.
  • [PB: For now the requirement is to allow moving from the last release forward. So Lois to Dylan, Dylan to Brian.]
• The update mechanism must preserve installed VMs and host configuration settings, such as wireless network settings, device admin password etc.
• For phase 1, the update mechanism must retrieve the update data from a https secure web server.
• For phase 1, it is acceptable to require the use of the dom0 CLI to apply an update, specifying the URL of the update data.
• The status report tool must accurately report which release is installed.

Mechanism

• Updates must be applied in a transactional manner: either all of the update is applied or none of it is.
• Metadata from each update must be recorded on the client, such that any subsequent status report will identify that the update has been applied.
• The update mechanism must not require an active network connection once the update package has been downloaded.
• Errors during application of the update must be reported.
• The update mechanism must be compatible with XenClient's use of TXT without triggering a tampering-detected event.
  • Two clients at the same release must have the same filesystem checksums, regardless of which release was originally installed on each client, which upgrades were applied and which update mechanisms were used.

Packaging

• An update must be distributable as a single file with a declared three-letter filetype suffix.
• It must be possible to securely verify whether an update originates from a proper source or not.

Phase 2

General

• For phase 2, it must be possible to apply an update using the Client UI by entering the URL of the update data.
  • [PB: This was a nice to have, so either allow upgrade via Client UI or just continue to support the CD based upgrade.]
  • For phase 2, it is not acceptable to require the use of the dom0 CLI to apply an update.
• Update from the Client UI must be possible regardless of whether the Client is registered to a Synchroniser.
• The Client UI must indicate which release is installed.
• The Client UI must report the download progress of the update data during transfer.
• The Client UI must prompt the user to reboot the host once application of the update has completed.

Modifications to support smaller packages ("surgical" updates)

General

• The update mechanism must be capable of applying a hotfix.
  • A hotfix is a targetted fix for a specific issue in a specific XenClient release.
  • Application of a hotfix must only be allowed if the client is at the correct release.
  • Application of a hotfix must not be prevented by the prior application of another hotfix.
  • Application of a hotfix must not undo the application of another hotfix.
• The client UI must indicate which hotfixes have been applied.
• The status report tool must accurately report which hotfixes have been applied.

Packaging

• The size of a hotfix must be proportional to the size of the change it carries.

Mechanism

• The update mechanism must be compatible with XenClient's use of TXT without triggering a tampering-detected event.
  • Two clients at the same release and with the same set of hotfixes applied must have the same filesystem checksums, regardless of which release was originally installed on each client, which upgrades were applied, which order the hotfixes were applied and which update mechanisms were used.

Modifications to support update control via Synchronizer

General

• It must be possible to apply an update via the Synchroniser to registered clients.
  • The Synchroniser must be able to function with clients with differing releases and different sets of hotfixes applied.
    • [Restrict combinations of Synchroniser and client release?]
      • [PB: The requirement I put in was starting with Brian we need to go one back. So Brian needs to support Dylan clients but not Lois clients. In the future we will likely need to do more but for now one back is fine.]
  • The Synchroniser must be aware of which release is installed on each client and which hotfixes have been applied.
  • It must be possible to upload client upgrades and hotfixes to the Synchroniser.
    • [PB: Would be good to define this a bit more. If we can do it I’d like to have the admin be able to use the Sync web UI and browse for a local file on their system to upload. Hit the upload button and get some basic progress and a message that the hotfix, upgrade is valid and who it comes from.]
  • It must be possible through the Synchroniser UI to target upgrades and hotfixes to specific clients.
    • The Synchroniser must be aware of which upgrades and hotfixes can be applied to each client.
    • [Ways to select clients?]
      • [PB: I spoke with you guys last time about doing device based selection but allowing the admin to derive a list of devices by entering a username or group name from a local user db or from AD.]
    • [Automatically applying upgrades/hotfixes to new clients?]
      • [PB: After the current environment gets upgraded then the admin will want to flip a switch and have it automatically upgrade any new clients added to the environment that are on an older version of the software.]
      • [PB: The Synchronizer will need the ability to report on the version of XenClient software that is running on each endpoint and provide a mechanism to target the over the air upgrades to a subset or the total population of XenClient endpoints. The administrator should have the ability to phase the upgrades of endpoints over a period of time and target specific systems, users, or groups of users for upgrades. The user and group targeting should include both local and Active Directory linked objects. The system should also have a mechanism to report on upgrade failures.]
  • The Synchroniser must be aware when a client update fails.
    • [PB: Needs to allow an admin to run a report and see which systems failed.]
    • [PB: In some future release we would want to send e-mail alerts, etc, But running a report is fine for now.]
  • The update mechanism must obtain the consent of the user on the client before the update proceeds.
  • The update mechanism must not require the device admin password to be entered on the client.
  • [PB: The update mechanism should allow the user to postpone an update for some reasonable period of time. We should map to the Windows update process. I think the options are 10 minutes, 30 minutes, 1 hour, 4 hours.]

Later work

General

• In the future we may need to expand the upgrade path so that you could go from a major release (XC 1.0 - Lois) or a minor release such as a service pack (XC 1.0.1 - Dylan) to the next major Brian (XC 2.0).
  • [PB: So in the example above for now they would need to be on the latest 1.0.x release to move up to the 1.1 but in the future we may ask for more options.]

Packaging

• Allow secure verification of updates from other sources, e.g. OEMs, partners

Future considerations

• Compatibility between hotfixes and OEM customisations.
• OEM customisations applied via hotfix mechanism?
• Interactions with supplemental packs.
• How to update tools installed in VMs?
  • [PB: For the first go around we could do alerts within the guest. But we have to figure out how to run the installer with elevated privileges as the regular Windows user might not have that level of access on a locked down business VM.]
• Use same mechanism for Kent/non-Kent variants? Difficult since creates a VM and hotfixes don't survive upgrades.
• Applying upgrades and hotfixes to Synchroniser?
• [Hotfix with prerequisite hotfixes?]
  • [PB: This is really up to you guys, I don’t have a problem with a hotfix requirement for another hotfix but it may get messy for our admins. It might be better to bundle any prereqs into the hotfix.]
• [Hotfix superseding another hotfix?]
  • [PB: This is really up to you. I think eventually we need this but in the early days we can likely just document that a certain hotfix supersedes another.]
• [Hotfix being incompatible with another hotfix?]
  • [PB: Could but I think it’s an edge case, I would design for adding this in the future.]

Updating via host installer

[Need to decide if this is necessary. Main motivation is to recover from problems where client won't boot.]

• It must be possible to apply an update from the host installer, either from optical media or via the network.
  • The host installer must require the device admin password to be entered on the client.