Самое полное руководство по работе с TweetDeck :: (12) Oleg Michman 25% �� 146% ☢️ ☣️⚠️ (@CASBT_UA) / Твіттер

 

Forwarding a video card to a virtual machine

By

 

iommu 

Habr

8 min

View Original

They say that modern hardware technologies to support virtualization (VT-d from Intel, IOMMU from AMD) allow a physical device on the PCI bus to be directly managed by a virtual machine. Including a video card.
Imagination draws the following configuration: a desktop server with a hypervisor, a guest user operating system is launched on it, which has access to the necessary input-output devices, one or two unpretentious servers as needed, and how many virtual machines are needed for inhuman experiments. We control the hypervisor through the console in the guest OS or remotely from a laptop, say.
Inspired by this picture, I decided to try it, but it turned out that the passthrough of the video adapter is not a completely trivial task. Only three months later, butting with iron and reading the forums, I managed to get a positive result. I tried VMware and Xen as a hypervisor . It only worked with Xen.

Short.
Iron:

• Материнская плата Gigabyte GA-Q67M-D2H-B3
  Прошивка BIOS — F5
  Setup: CPU->Intel Virtualization Technology=ON
  Setup: Chipset->North Bridge->VT-d=ON
• Intel Core i5 2500 3.3 GHz processor
• Memory 16 GB
• ATI Radeon HD 3470 in the first PCIe slot, used by the hypervisor
• ATI Radeon HD 3450 in the second PCIe slot, given to the guest OS
• Intel network adapter in PCI slot

Software:

• XCP (Xen Cloud Platform) 1.0 (XenServer build 42052c)
• Citrix XenCenter for management
• Windows 7 64 bit as guest OS, ATI Catalyst 12.1

At first, I experimented with VMware vSphere 5.0 for a long time. Actually, the hardware configuration was selected exactly for it. On the way, a number of interesting details were revealed: for example, VT-d must be supported by the processor (they write that processors with the K index are not suitable), and by the chipset and the motherboard. The problem with video cards is that it is known that with the majority this trick does not work, with some (rather short list) some succeed, others do not. A long and meaningful (although not too joyful) discussion was here:
VMware Communities: VMDirectPath and ATI Radeon . The Radeon 3450 was perhaps in the favorites as one of the most thrown cards.

I went through a decent amount of different combinations of iron. The competition included two motherboards, three video cards plus SandyBridge Integrated Video (IGD), three network adapters, and one processor. Several times I gave up these fruitless attempts for a week or two, then I came up with some options. On the way, there was one moment when it almost succeeded: the virtual machine correctly identified the monitor, but it did not go further. I rested on the fact that the card seemed to be normally thrown into the virtual machine, and in the device manager it showed exactly, but the Catalist stubbornly refused to deal with it. The card is alive, but does not work.

You could try a lot of other things: Windows XP and Linux as guest systems (installed Windows 7 in 32 and 64-bit versions), get another video card ... In the end, he spat and decided to go from the other end, trying another hypervisor. Without further ado, I took what was in sight: Xen as part of the Xen Cloud Platform (XCP).
XCP delivered flawlessly.

For a while, I puzzled the question: how to steer this system? I mean, should there be some kind of management console, preferably for Windows? After digging around for half a day with the conditionally standard OpenXenManager, I came to the conclusion that either skis do not go, or this cross-platform tool does not live on Windows. Once or twice she connected to the server, but died somewhere in the process of work, the rest of the times it hung dully when connected, merging an unrecoverable stream of exceptions into the Python console.
Fortunately, a broader look at the surrounding Internet revealed to me that Citrix XenCentercan perfectly steer opensource Xen, and he himself is completely free. True, when connected, he screams that in N days your server will expire the Evaluation period, but knowledgeable people write that he simply does not know about the opensource edition of the server, but in fact everything will work.
XenCenter allows you to create-enable-disable virtual machines, and device forwarding must be configured from the sysadmin-friendly command line interface.

Contrary to expectations, no problems happened here. I did everything according to the manual , and it was enough for him alone. People complain that there is little documentation on Xen. So another time and it's good that it is not enough, if this is enough. How much I have read about vSphere, and it’s all to no avail ... However, I don’t want to say bad words about vSphere. Under it, on the other hand, the iron tuned in so that Xen flew straight with a whistle.
So, using XenCenter, I organized a virtual machine with two cores and 4 GB of memory, rolled the seventh 64-bit Windows there and went to forward it.

In accordance with the manual, we are correcting /boot/extlinux.conf, inserting a line "iommu=1 iommu_inclusive_mapping=1"after each occurrence . "/boot/xen.gz"
Execute extlinux /boot.
We pcibackskip the steps related to the module - they write that in the six it is already compiled into the kernel.
We make xe vm-list and find the uuid of our virtual machine, mine. uuid=d103a91d-5c38-844f-14d5-64b3c495eb08
Execute the command lspciand find our card in the output, for example 02: 00.0 VGA compatible ..., 02: 00.1 Audio ... (the two surprisingly corresponds to the number of the slot where the card is inserted).
We write a one-line script like
xe vm-param-set other-config:pci=0/0000:02:00.0,0/0000:02:00.1 uuid=d103a91d-5c38-844f-14d5-64b3c495eb08
I have a sound on my card, in addition to the video adapter - therefore, remembering the rake that I stepped on in vSphere, I add both devices: 0/0000: 02: 00.0, 0/0000: 02: 00.1.
We execute the script. For controlxe vm-param-list uuid=d103a91d-5c38-844f-14d5-64b3c495eb08 | more- indeed, we
stop and restart the virtual machine (they write that this is exactly how it is, and not a reboot - I will not check it again).
On the first attempt, the card is in the first PCIe slot (01: 00.0, 01: 00.1) and is used by the hypervisor by default. After restarting the virtual machine, the monitor goes out.
In XenCenter (from a laptop) we go into the virtual console and after logging into Windows we see that it is asking for a reboot. A sign that she has found a new device. We will not refuse her. Reboot. Indeed, the Device Manager has a new video adapter Radeon 3450 with the Microsoft WDDM 1.1 driver. From previous experience it is known that the driver needs a native one. I download and install a fresh ATI Catalist 12.1, which after installation, as usual, asks for a reboot. Reboot ... oh shit. The first attempt is covered with a copper basin: the hypervisor falls. Yes ... vSphere in such a situation won a convincing victory over the virtual machine, arranging BSOD for it.
We bypass the host and, on the recommendation of the best dog breeders, see what the team writes to us
dmesg. She writes, among other things:

pciback 0000:01:00.1: secondary bus reset failed for device — all functions need to be co-assigned — err: 6
pciback 0000:01:00.1: FLR functionality not supported; attempts to use secondary bus reset unsuccessful;
pciback 0000:01:00.1: FLR not performed for device


It seems that the transfer of the card to the hot one is out of our reach. OK. Let's give the hypervisor our VGA adapter, since now I have enough video cards. We rearrange the Radeon 3450 into the second slot, put the 3470 lying around in the first one. Attach a monitor to each card. We turn on the host, start the virtual machine. Windows asks for a reboot after changing the configuration. Reboot. Login ...

In the XenCenter console, the splash screen freezes, and I do not immediately realize that the right monitor is glowing blue

It did happen.
In total, it grew on Xen in 3 days (after 3 months of practicing on VMware).

I'm logged in. The picture on the monitor is the most ordinary, without any peculiarities. The resolution is 1920x1200. Not stupid (although I did not run tests). YouTube videos play fine.

On this joyous note, I will complete my truthful report, and I myself will be engaged in forwarding the keyboard, mouse and other peripherals.

Update: I
forwarded the keyboard and mouse, I write from the virtual machine under Win7. Nothing here, you can live.
Performance Index 3.5

I haven't even turned off Aero and other visual effects that are harmful to our business yet, but it works smartly.

I did the USB forwarding in a rude and cynical way:
xe vm-param-set other-config:pci=0/0000:02:00.0,0/0000:02:00.1,0/0000:00:1a.0,0/0000:00:1d.0 uuid=d103a91d-5c38-844f-14d5-64b3c495eb08
That is, I donated USB controllers to the virtual machine . On the other hand, Xen will do without them for now.

What else? Delivered XenCenter, Xen is normally administered (who would doubt it). An external USB drive, of course, also attaches normally. Now you need to figure out how to forward the CD-ROM.

Unfortunately, nothing worked for me with USB Passthrough , as well as with the Xen VGA Passthrough ( Scraelos ) justly mentioned in the comments , because there is no file "/ etc / xen / cfgfile" under XCP. How to register the necessary settings with the help xe- I did not figure it out. If the Xen experts can help, I would be very grateful.

Update: ToDo:

• Throw CD-ROM (-)
• Configure static IP (+)
• Test performance (partially done)
• Try to transplant dom0 to IGD, one PCIe slot will become free (-)
• Try to transplant dom0 to the onboard NIC, the PCI slot will become free (+)
• Try to sell dom0 keyboard to PS / 2
• Organize a file system for data exchange between guest operating systems
• Organize switching between guest operating systems (scripts, probably)

Update 02/06/2012:
dom0 refused to transfer to IGD categorically. In addition, I tried to forward the primary VGA adapter again - to no avail. Returned to the previous adapter configuration.
Rested in a problem with forwarding CD (DVD) -Writer. CD-ROM is forwarded normally, but only RO, but I need RW. In this regard, I found 2 recommendations: plug in and forward a separate SATA controller and use a USB CD / DVD-Writer (fortunately, USB is forwarded). Unfortunately, even these (IMHO, crutch) solutions have not worked for me at the moment. The controller refused to be forwarded. An attempt to connect a standard SATA CD / DVD drive via a USB-SATA adapter did not lead to anything good. I continue my experiments.

Просмотры: