New Linux Driver for Argo

OXT-1473

There are multiple reference code bases for a Linux driver for Argo.

OpenXT's current Linux Argo driver took the OpenXT v4v driver as a starting point, updated to use Argo interfaces.
The code for this driver, "argo-linux", is in this OpenXT repository: https://github.com/OpenXT/linux-xen-argo
It presents a character device that has a socket-like interface implemented with ioctls.
Eric implemented a new vsock driver with datagram support, "vsock-argo", merged from PR#1 to the same OpenXT linux-xen-argo repository:
https://github.com/OpenXT/linux-xen-argo/tree/master/vsock-argo
HP/Bromium’s latest Open Source uXen VM support software for Linux guests is available here:
https://www.bromium.com/wp-content/uploads/2019/06/uxen-vmsupport-linux-4.1.7.zip
It includes two modern v4v drivers that work together:
1. a v4v library driver, uxenv4vlib, which:
  1. registers for the interrupt line
  2. performs basic ring operations
  3. provides an interface to kernel drivers for querying for the space available in a remote ring ("uxen_v4v_notify_space").
2. a simple v4v vsock driver, v4vvsock, which:
  1. registers a vsock driver (AF_vsock)
  2. provides a datagram interface only. It is simple - eg. provides no ioctls.
Previous attempts at cleaning up v4v:

Eric's vsock
Eric's AF_V4V
Daniel's cleanup

Consolidated Design Notes

( please see the comments on OXT-1473 where individual inputs have been given )

On the driver:

Argo is now in upstream Xen and correspondingly the Linux driver needs to be prepared for upstreaming to the Linux kernel
- vsock is upstream in Linux, there are transports for virtio and VMware, https://vmsplice.net/~stefan/stefanha-kvm-forum-2015.pdf
- vsock can provide the socket behavior, which should minimize resistance in getting the driver suite upstreamed
The driver should expose two separate interfaces: a vsock interface and a character device
- The driver should be built as a common argo-core – along the lines of uXen's uxenv4vlib – with both char and vsock drivers then leveraging argo-core
- Using the existing socket-like interface on the character device does not depend on availability of a network stack
  - Potentially useful for minimal services VMs
  - Change requested: "The char driver behavior should be reduced back to that of a conventional Linux char driver"
- Xen’s Argo interface will change to support message labeling
  - This will enable the kernel to access the XSM/Flask SID label with the arriving data that it is associated with
  - OXT-1503: Argo peer labelling for access control
  - Test coverage for this will be required, so a method of enabling test cases to access and validate it needs to be determined
  - Kernel use of the label: guidance in Smalley presentation on OpenXT's access control at OpenXT Summit 2016
  - Should the label be surfaced to userspace?
- How should either the vsock or chardev interfaces support Argo-specific functionality?
  - ie. Is the chardev interface the correct interface between userspace and the kernel to provide support for Argo-specific, non-traditional character device, functionality?
    - eg. Configuring ring size
The proposal for an Argo name service will need consideration of how names interact with this driver.
The Argo firewall is currently not in upstream Xen, pending implementation of bi-directional connection tracking in OpenXT before upstreaming. The Linux driver is used to access and configure the firewall.
- The firewall functions should be moved into a separate Linux driver.

Summary: the driver will need to provide:

The standard vsock interface
A character device interface
- either: continuing to provide a socket-like interface for the non-networking use case
- or: reduced to that of a conventional Linux char driver
- or: both, but KCONFIG applied to separate the basic char driver functions from the optional socket-like interface support
Either vsock or chardev extended interface
- to access Argo-specific functions (eg. configure ring size; eg. manage firewall rules)
- to provide MAC context data, eg. 32-bit field for the SID with arriving message data if decided that userspace should have this context presented this way with arriving data

On userspace:

libargo can provide a single userspace library that is able to utilize either the chardev or vsock kernel interfaces
To use the vsock driver interface, userspace apps would typically require modification to be vsock-aware
- Some projects already support vsock, e.g. python.
Continued use of LD_PRELOAD with the existing Argo interposer library is unwanted
- For existing apps that use network protocols, socat (socket proxy) performs a similar role for vsock as the interposer library does for the current Argo driver
  - socat should have a smaller attack surface since it does not use LD_PRELOAD
- UIVM has many uses of LD_PRELOAD
  - These userspace apps could be modified to support vsock, and the patches submitted to upstream projects

Testing

For the current chardev driver: PR#4 on the OpenXT bats-suite repository contains basic initial BATS test cases for Argo.
For the vsock-argo driver: has a test utility "hatch" to exercise datagram transfers over vsock through Argo.
- "Simple test tool to send/recv things over vsock through Argo and ease testing. It tries to mimic netcat usage to not look too unfamiliar. socat should be able to achieve the same, with a steeper learning curve. This will be made easier once STREAM communication are implemented I would think."
For the uXen v4v driver: a very simple C program is provided that runs two threads, with a bind, a connect and sending data from one to the other.

Copyright 2019 by BAE Systems. This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.