GPU PT broken

Description

Passing through a nVidia M2000 on XE3 to guest fails. Build 6524

{{Apr 22 21:21:52.131253 VM win1809-dm (24): [ 7.777865] pcifront pci-0: PCI host bridge to bus 0000:01
Apr 22 21:21:52.131280 VM win1809-dm (24): [ 7.777871] pci_bus 0000:01: root bus resource [io 0x0000-0xffff]
Apr 22 21:21:52.131285 VM win1809-dm (24): [ 7.777879] pci_bus 0000:01: root bus resource [mem 0x00000000-0x7fffffffff]
Apr 22 21:21:52.131289 VM win1809-dm (24): [ 7.777886] pci_bus 0000:01: root bus resource [bus 00-ff]
Apr 22 21:21:52.131943 kernel:[ 982.222857] pciback 0000:01:00.0: registering for 24
Apr 22 21:21:52.131952 kernel:[ 982.226254] pciback 0000:01:00.0: enabling device (0000 -> 0003)
Apr 22 21:21:52.131953 kernel:[ 982.226339] Already setup the GSI :16
Apr 22 21:21:52.142017 kernel:[ 982.226627] Already setup the GSI :16
Apr 22 21:21:52.142018 kernel:[ 982.226867] Already setup the GSI :16
Apr 22 21:21:52.142019 kernel:[ 982.227095] Already setup the GSI :16
Apr 22 21:21:52.142020 kernel:[ 982.227366] Already setup the GSI :16
Apr 22 21:21:52.142020 kernel:[ 982.227534] Already setup the GSI :16
Apr 22 21:21:52.345345 qmp_helper: qmp-helper [../../git/qmp_helper/src/qmp_helper.c:main:342] (stubdom-24) Accepted the connection fd: 12, telling qemu.
Apr 22 21:21:52.345610 VM win1809-dm (24): qemu-system-i386: v4v_recvfrom() returned V4V_MAGIC_CONNECT, connecting.
Apr 22 21:21:52.346338 xl: [12246] libxl_qmp.c:292:qmp_handle_error_responseomain 23:received an error message from QMP server: Could not open '/sys/bus/pci/devices/0000:01:00.0/config': No such file or directory
Apr 22 21:21:52.346354 xl: [12246] libxl_pci.c:1318:libxl__add_pcidevsomain 23:libxl_device_pci_add failed: -3
Apr 22 21:21:52.346361 xl: [12246] libxl_create.c:1620:domcreate_attach_devicesomain 23:unable to add pci devices
Apr 22 21:21:52.346396 qmp_helper: qmp-helper [../../git/qmp_helper/src/qmp_helper.c:qmph_unix_to_v4v:114] (stubdom-24) read(unix_fd) recieved EOF, telling qemu.
Apr 22 21:21:52.346430 qmp_helper: qmp-helper [../../git/qmp_helper/src/qmp_helper.c:qmph_accept_unix_socket:195] (stubdom-24) Waiting for connection on unix socket
Apr 22 21:21:52.346554 VM win1809-dm (24): qemu-system-i386: v4v_recvfrom() returned V4V_MAGIC_DISCONNECT, closing.
Apr 22 21:21:52.346612 qmp_helper: qmp-helper [../../git/qmp_helper/src/qmp_helper.c:main:342] (stubdom-24) Accepted the connection fd: 12, telling qemu.
Apr 22 21:21:52.346727 VM win1809-dm (24): qemu-system-i386: v4v_recvfrom() returned V4V_MAGIC_CONNECT, connecting.
Apr 22 21:21:52.347443 qmp_helper: qmp-helper [../../git/qmp_helper/src/qmp_helper.c:qmph_unix_to_v4v:114] (stubdom-24) read(unix_fd) recieved EOF, telling qemu.
Apr 22 21:21:52.347472 qmp_helper: qmp-helper [../../git/qmp_helper/src/qmp_helper.c:qmph_accept_unix_socket:195] (stubdom-24) Waiting for connection on unix socket
Apr 22 21:21:52.347482 qmp_helper: qmp-helper [../../git/qmp_helper/src/qmp_helper.c:main:342] (stubdom-24) Accepted the connection fd: 12, telling qemu.
Apr 22 21:21:52.347563 VM win1809-dm (24): qemu-system-i386: v4v_recvfrom() returned V4V_MAGIC_DISCONNECT, closing.
Apr 22 21:21:52.347617 VM win1809-dm (24): qemu-system-i386: v4v_recvfrom() returned V4V_MAGIC_CONNECT, connecting.
Apr 22 21:21:52.348081 qmp_helper: qmp-helper [../../git/qmp_helper/src/qmp_helper.c:qmph_unix_to_v4v:110] (stubdom-24) ERROR read(unix_fd) failed (Connection reset by peer) - -1.
Apr 22 21:21:52.348088 qmp_helper: qmp-helper [../../git/qmp_helper/src/qmp_helper.c:main:356] (stubdom-24) exiting...
Apr 22 21:21:52.354323 VM win1809-dm (24): config qemu network with xen bridge for vif23.0-emu
Apr 22 21:21:52.354332 VM win1809-dm (24): Bridge stubdombr0 already exists, strange (invoked for vif23.0-emu)
Apr 22 21:21:52.361770 VM win1809-dm (24): Adding eth0 and vif23.0-emu to stubdombr0
Apr 22 21:21:52.363050 VM win1809-dm (24): device eth0 is already a member of a bridge; can't enslave it to bridge stubdombr0.
Apr 22 21:21:52.366079 VM win1809-dm (24): device vif23.0-emu is already a member of a bridge; can't enslave it to bridge stubdombr0.
Apr 22 21:21:52.369543 VM win1809-dm (24): [ 8.016098] stubdombr0: port 2(vif23.0-emu) entered disabled state
Apr 22 21:21:52.369550 VM win1809-dm (24): [ 8.016132] device vif23.0-emu left promiscuous mode
Apr 22 21:21:52.369554 VM win1809-dm (24): [ 8.016137] stubdombr0: port 2(vif23.0-emu) entered disabled state
Apr 22 21:21:52.647270 VM win1809-dm (24): [ 8.293812] pci 0000:01:00.0: vgaarb: setting as boot VGA device
Apr 22 21:21:52.647280 VM win1809-dm (24): [ 8.293821] pci 0000:01:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
Apr 22 21:21:52.649512 VM win1809-dm (24): [ 8.296064] pcifront pci-0: claiming resource 0000:01:00.0/0
Apr 22 21:21:52.649522 VM win1809-dm (24): [ 8.296073] pcifront pci-0: claiming resource 0000:01:00.0/1
Apr 22 21:21:52.649526 VM win1809-dm (24): [ 8.296079] pcifront pci-0: claiming resource 0000:01:00.0/3
Apr 22 21:21:52.649529 VM win1809-dm (24): [ 8.296084] pcifront pci-0: claiming resource 0000:01:00.0/5
Apr 22 21:21:52.649533 VM win1809-dm (24): [ 8.296089] pcifront pci-0: claiming resource 0000:01:00.0/6
Apr 22 21:21:52.649556 VM win1809-dm (24): [ 8.296148] pci 0000:01:00.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
Apr 22 21:21:52.651891 surfman: Info: DM disconnected. domid 23
Apr 22 21:21:52.651949 input_server: Info:../git/server.c:rpc_disconnect:88:input_server: DM 23 disconnected
Apr 22 21:21:52.651963 input_server: Info:../git/domains.c:domain_gone:393:cleaning up domain 23
Apr 22 21:21:52.654841 VM Network (1): [ 966.756297] brbridged: port 5(vif24.0) entered disabled state
Apr 22 21:21:52.656665 tapdisk[12256]: disconnecting domid=24, devid=768
Apr 22 21:21:52.671577 VM Network (1): [ 966.773043] brbridged: port 5(vif24.0) entered disabled state
Apr 22 21:21:52.671726 VM Network (1): [ 966.773179] brbridged: port 5(vif24.0) entered disabled state
Apr 22 21:21:52.680884 tapback[12259]: backend.c:1114 domain removed, exit
Apr 22 21:21:52.682343 tapback[1137]: backend.c:807 master restart not yet implemented, ignoring domain 1
}}

Environment

M2000 on Dell XE3

Validation Steps

1. Pass through GPU to guest.
2. Start VM.
3. VM fails to start.
4. Errors above.

Activity

Show:
Rich Persaud
April 23, 2019, 11:11 PM

This ticket is for Quadro 2000, which is supported by Nvidia for GPU PT. is for a non-Quadro Nvidia GPU, which is not supported by Nvidia for GPU PT.

Eric Chanudet
April 23, 2019, 11:13 PM

, Chris referenced the NVS 310 explicitly in his comment above, which is why I mentioned that ticket.

Joel Upham
April 24, 2019, 3:35 AM
Edited

In my experience NVS series have issues with PT to guests, if you can get me the patched version of libxl I can try on the XE3 with the quadro M2000.

Chris Rogers
April 24, 2019, 1:36 PM

It's the only card I have at my office right now, I'll send Joel the patched libxl to see if his Quadro works.

Andrew Jones
May 13, 2019, 10:04 PM

Was able to repro the issue in 6524, then successfully pass-through with the same system setup in 6552. Can confirm that update fixes this issue.

During testing of this ticket, encountered significant issues with PCI cards and their behaviour with OXT. Resulted in creating OXT-1600, OXT-1601,

Fixed

Assignee

Unassigned

Reporter

Joel Upham

Labels

None

QA Assignee

None

QA Image URL

None

Epic Link

Fix versions

Affects versions

Priority

Blocker
Configure