Determine and ensure that the functionality of OpenXT has not been compromised by the staggering changes that the product has been subjected to. Validation of the entire product needs to be determined through the testing of these components.
Regressions will be determined by evaluating the system from previous baselines of stable OpenXT and downstream derivative products. Every major part of OpenXT has been changed, including:
Base functionality of installing the system, installing guest VMs, and having a workable UI, as well as testing out command line options, should allow us to get a handle on how broken the system may be.
Stress and performance testing is vital to the stability of the system. Many problems that normally do not appear in normal testing will crop up relatively quickly in these tests and will help weed out any unseen issues.
With the sudden change of so many elements, it is important to look at what changes have occurred for the better and utilize the new technologies that the upgrades have provided the system.
The current plan is for two AIS Systems Engineers to test the OpenXT platform, taking two weeks to complete the test execution plan.
Hardware resources:
Bugs are being logged against the epic OXT-1568. Testing has started and the chart is being updated daily, as these tests are the bare minimum and more can be added.
| Test Desc | Platform | Host Boot | TPM v | OXT v | OS Version | OS Boot | Result | Notes/Test Data |
|---|---|---|---|---|---|---|---|---|
| Install Windows guest VM with Tools. | Dell Latitude 7490 | UEFI | 2.0 | Stable 9 6524 | Windows 10 1803 | Legacy | PASS | |
| Install Windows guest VM with Tools. | Dell XE3 | UEFI | 2.0 | Stable 9 6524 | Windows 10 1803 | UEFI | FAIL | Auto-start works, memory set to 8GB, 4vcpus, tools cause black screen. Issue: OXT-1572 |
| Install Windows guest VM with Tools. | Dell Latitude 7490 | UEFI | 2.0 | Stable 9 6524 | Windows 10 1809 | Legacy | PASS | |
| Install Windows guest VM with Tools. | Dell XE3 | UEFI | 2.0 | Stable 9 6524 | Windows 10 1809 | UEFI | FAIL | Tools fails with a black screen. Issue: OXT-1572 |
| Install RHEL guest VM. | NA | NA | NA | NA | RHEL 7.6 | Legacy | ||
| Install RHEL guest VM. | Dell XE3 | UEFI | 2.0 | Stable 9 6524 | RHEL 7.6 | UEFI | PASS | RHEL 7.6 installed using text installer, visual installer failed. Tools untested. |
| Configure guest VM in multiple configurations. Including adding multiple hard drives, vCPU, memory, boot order, icon, and all advanced configurations. | NA | NA | NA | NA | Windows 10 1803 | Legacy | ||
| Configure guest VM in multiple configurations. Including adding multiple hard drives, vCPU, memory, boot order, icon, and all advanced configurations. | NA | NA | NA | NA | Windows 10 1803 | UEFI | ||
| Configure guest VM in multiple configurations. Including adding multiple hard drives, vCPU, memory, boot order, icon, and all advanced configurations. | NA | NA | NA | NA | Windows 10 1809 | Legacy | ||
| Configure guest VM in multiple configurations. Including adding multiple hard drives, vCPU, memory, boot order, icon, and all advanced configurations. | NA | NA | NA | NA | Windows 10 1809 | UEFI | ||
GPU pass through testing:
| NA | NA | NA | NA | Windows 10 1803 | Legacy | BLOCKED | Issue OXT-1571 |
GPU pass through testing:
| Dell XE3 | UEFI | 2.0 | Stable 9 6524 | Windows 10 1803 | UEFI | FAIL | GPU PT fails to work on the M2000. Issue OXT-1571 |
GPU pass through testing:
| NA | NA | NA | NA | Windows 10 1809 | Legacy | BLOCKED | Issue OXT-1571 |
GPU pass through testing:
| Dell XE3 | UEFI | 2.0 | Stable 9 6524 | Windows 10 1809 | UEFI | FAIL | GPU PT fails to work on the M2000. Issue OXT-1571 |
PCI pass through testing:
| NA | NA | NA | NA | Windows 10 1803 | Legacy | ||
PCI pass through testing:
| Dell XE3 | UEFI | NA | 2.0 | Windows 10 1803 | UEFI | FAIL | USB PCIe device causes screen to go to static. Issue OXT-1570 |
PCI pass through testing:
| NA | NA | NA | NA | Windows 10 1809 | Legacy | ||
PCI pass through testing:
| Dell XE3 | UEFI | NA | 2.0 | Windows 10 1809 | UEFI | FAIL | USB PCIe device causes screen to go to static. Issue OXT-1570 |
UIVM: Test all buttons for correct functionality that are displayed and sub options.
| NA | NA | NA | NA | NA | NA | ||
| Network Configurations will be tested, making sure functions in the applet work as needed. Test DHCP and static setups. | NA | NA | NA | NA | NA | NA | ||
| Measured launch measures the correct PCRs for the Host system. PCR 15,16,17,18,19, at minimum. | NA | Legacy | 1.2 | NA | NA | NA | ||
| Measured launch measures the correct PCRs for the Host system. PCR 4,5,6,7,8,15,16,17,18,19. | NA | UEFI | 2.0 | NA | NA | NA | ||
| Measured launch measures the correct PCRs for the Host system. PCR 15,16,17,18,19, at minimum. | NA | Legacy | 2.0 | NA | NA | NA | ||
| Forward seal works. | NA | UEFI | 2.0 | NA | NA | NA | ||
| Forward seal works. | NA | Legacy | 1.2 | NA | NA | NA | ||
| Forward seal works. | NA | Legacy | 2.0 | NA | NA | NA | ||
| Upgrade to self. | NA | Legacy | 1.2 | NA | NA | NA | ||
| Upgrade to self. | NA | Legacy | 2.0 | NA | NA | NA | ||
| Upgrade to self. | NA | UEFI | 2.0 | NA | NA | NA | ||
Performance testing of NDVMs will include:
| NA | NA | NA | NA | NA | NA | ||
Guest Stress testing will include running multiple programs for many hours:
| NA | NA | NA | NA | NA | NA | ||
| Host Stress test will include starting and stopping guest VMs for up to 120 hours. | NA | NA | NA | NA | NA | NA | ||
| Other Ad-hoc testing to be documented. TBD. |
Adding this area for bugs and other information found during testing that occured but where not easily reproduced. Bugs may not exist in Jira, but areas that should be noted.