Stable-9 rc1 Test Matrix (results)

Owned by Nicholas Tsirakis
Last updated: Jul 22, 2019
Hardware used:
  • Dell Optiplex 7040
Guest VM OSs under Tests:
  • Windows 10 1809
  • Windows 10 1803
  • RHEL 7.5

Test Matrix

Test DescPlatformHost BootTPM vOXT vOS VersionOS BootResultNotes/Test Data
Install OpenXT with MLE - UEFI HP PlatformDell Optiplex 7040UEFI
Stable 9 6627N/AN/APASS
Install OpenXT no measured launch - UEFI HP PlatformDell Optiplex 7040UEFI
Stable 9 6627N/AN/APASS
Install Windows guest VM with Tools.Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10  1803LegacyPASS
Install Windows guest VM with Tools.Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10 1803UEFIPWEWon't fix: black screen after installing tools
Install Windows guest VM with Tools.Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10 1809LegacyPWEWon't fix: black screen after installing tools
Install Windows guest VM with Tools.Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10 1809UEFIPWEWon't fix: black screen after installing tools
Install RHEL guest VM.Dell Optiplex 7040UEFI2.0Stable 9 6627RHEL 7.5LegacyPASS


Install RHEL guest VM.Dell Optiplex 7040UEFI2.0Stable 9 6627RHEL 7.5UEFIPWEWon't fix: tools not working
Configure guest VM in multiple configurations.  Including adding multiple hard drives, vCPU, memory, boot order, icon, and all advanced configurations.Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10 1803LegacyPWE

Isolation - OXT-1577

Configure guest VM in multiple configurations.  Including adding multiple hard drives, vCPU, memory, boot order, icon, and all advanced configurations.Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10 1803UEFIPASS
Configure guest VM in multiple configurations.  Including adding multiple hard drives, vCPU, memory, boot order, icon, and all advanced configurations.Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10 1809LegacyPASS


Configure guest VM in multiple configurations.  Including adding multiple hard drives, vCPU, memory, boot order, icon, and all advanced configurations.Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10 1809UEFIPASS

GPU pass through testing:

  • Performance testing running GPU benchmark such as Heaven (UNIGINE) for at least one hour
  • Ad hoc testing
  • Memory Hole analysis
Dell Optiplex 7040NANAStable 9 6627Windows 10 1803LegacyPASS


GPU pass through testing:

  • Performance testing running GPU benchmark such as Heaven (UNIGINE) for at least one hour
  • Ad hoc testing
  • Memory Hole analysis
Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10 1803UEFIPASS


GPU pass through testing:

  • Performance testing running GPU benchmark such as Heaven (UNIGINE) for at least one hour
  • Ad hoc testing
  • Memory Hole analysis
Dell Optiplex 7040NANAStable 9 6627Windows 10 1809LegacyPASS


GPU pass through testing:

  • Performance testing running GPU benchmark such as Heaven (UNIGINE) for at least one hour
  • Ad hoc testing
  • Memory Hole analysis
Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10 1809UEFIPASS


PCI pass through testing:

  • Ad hoc testing
  • Memory Hole analysis
Dell Optiplex 7040NANAStable 9 6627Windows 10 1803LegacyPASS


PCI pass through testing:

  • Ad hoc testing
  • Memory Hole analysis
Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10 1803UEFIPASS


PCI pass through testing:

  • Ad hoc testing
  • Memory Hole analysis
Dell Optiplex 7040NANAStable 9 6627Windows 10 1809LegacyPASS


PCI pass through testing:

  • Ad hoc testing
  • Memory Hole analysis
Dell Optiplex 7040UEFI2.0Stable 9 6627Windows 10 1809UEFIPASS


UIVM: Test all buttons for correct functionality that are displayed and sub options.

  • Settings
  • VM Details
  • Status
  • Services
  • Devices
  • Install VM
  • Power Button
  • etc
Dell Optiplex 7040UEFINAStable 9 6627Windows 10 1809LEGACYPWE

Power: FAIL - OXT-1591

Info: PWE - Many issues with Status Report, no tickets submitted

Settings: PWE

Audio: Auto-mute Mode inconclusive

Audio: IEC958 settings inconclusive

Audio: Mic Mute LED Mode: inconclusive

Audio: Unable to disable ALL mic options simultaneously, one will also reset to "on."  No documentation to tell if feature or bug

Software Update: inconclusive

Network Configurations will be tested, making sure functions in the applet work as needed. Test DHCP and static setups.Dell Optiplex 7040NANAStable 9 6627NANAPASS
Measured launch measures the correct PCRs for the Host system.  PCR 15,16,17,18,19, at minimum.Dell Optiplex 7040Legacy1.2Stable 9 6627NANAPASS
Measured launch measures the correct PCRs for the Host system.  PCR 4,5,6,7,8,15,16,17,18,19.Dell Optiplex 7040UEFI2.0Stable 9 6627NANAPASS
Measured launch measures the correct PCRs for the Host system.  PCR 15,16,17,18,19, at minimum.Dell Optiplex 7040Legacy2.0Stable 9 6627NANAPASS
Forward seal works.Dell Latitude 7490UEFI2.0Stable 9 6627NANAPASS
Forward seal works.Dell Optiplex 7040Legacy1.2Stable 9 6627NANAPASS
Forward seal works.Dell Optiplex 7040Legacy2.0Stable 9 6627NANAPASS
Upgrade to self.Dell Optiplex 7040Legacy1.2Stable 9 6627NANAPASS
Upgrade to self.Dell Optiplex 7040Legacy2.0Stable 9 6627NANAPASS
Upgrade to self.Dell Optiplex 7040UEFI2.0Stable 9 6627NANAPASS

Performance testing of NDVMs will include:

  • pinging a gateway and other addresses to look at latency
  • large file (100+GB) downloads
  • wireshark analysis if needed
  • Add-on vs. integrated NIC
Dell Optiplex 7040NANAStable 9 6627NANANOT TESTING

Guest Stress testing will include running multiple programs for many hours:

  • 72 hours of HeavyLoad stressing memory and CPU, Prime95, and a video being run.
  • 120 hours of a guest running a video and refreshing a webpage.
Dell Optiplex 7040NANAStable 9 6627NANANOT TESTING
Host Stress test will include starting and stopping guest VMs for up to 120 hours.Dell Optiplex 7040NANAStable 9 6627NANANOT TESTING