Passing a GPU to a Proxmox VM with PCI Passthrough

Enabling PCI Passthrough

  1. Log into ProxMox VE either via SSH or the web based shell
  2. Run the following commands
    # edit grub configuration to enable IOMMU
    nano /etc/default/grub
  3. Press CTRL+W and search for GRUB_CMDLINE_LINUX_DEFAULT=
  4. Add the following value to GRUB_CMDLINE_LINUX_DEFAULT
    1. If running an Intel CPU, intel_iommu=on

      GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"

    2. If running an AMD CPU, amd_iommu=on

      GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"

  5. Press CTRL+O, Enter, CTRL+X to write the changes
  6. Continue with the following commands
    # update grub with config change
    update-grub
    # add vfio modules
    nano /etc/modules
  7. Add the following to the bottom of the file

    vfio
    vfio_iommu_type1
    vfio_pci
    vfio_virqfd

  8. Press CTRL+O, Enter, CTRL+X to write the changes
  9. Continue with the following commands
    # IOMMU interrupt mapping
    echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
    echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
    # blacklist drivers
    echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
    echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
    echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
    # list pci devices
    lspci -v
  10. Scroll through the output and find the GPU (usually a video and audio device associated with it) and make note of the device identifier (ex 01:00.0, 01:00.1)
  11. Continue with the following commands
    # output the vendor ID(s) of the devices
    # replace the device identifier (01:00) as needed
    lspci -n -s 01:00
  12. Make note of the vendor ID(s) output in the format abcd:1234
  13. Continue with the following commands
    # replace the vendor ids from the output of the previous step as needed
    echo "options vfio-pci ids=1002:6611,1002:aab0 disable_vga=1"> /etc/modprobe.d/vfio.conf
    # update initramfs
    update-initramfs -u
    # reboot proxmox host
    reboot now

Passing GPU to A VM

  1. Log into the ProxMox web UI
  2. Expand the target node and select the target VM from the left navigation menu
  3. Select Hardware from the left sub-navigation menu
  4. Click Add > PCI Device
  5. Select the GPU from the listing
    NOTE: I have found that there is some trial and error required to get the GPU to work properly depending on the guest operating system. If you find that the GPU isn't working with all options checked, stop the VM, and edit the PCI hardware, unchecking one option at a time to see if you can find a working combination
  6. Select Display > Set the dropdown to none (none) > Click OK
  7. If you intend to use the VM at the console (not via RDP or VNC), you will need to passthrough a USB keyboard and mouse by clicking Add > USB Device > Select the USB device(s) or port(s) to pass to the VM
  8. Click Start at the top right of the page
  9. Watch the output of the GPU on an external display
    If there is no output from the GPU, stop the VM and adjust the PCI hardware options as noted above
  10. Once the VM has booted and is outputting to the GPU, install drivers for the GPU as necessary

Source: https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/
Documentation: https://pve.proxmox.com/wiki/Pci_passthrough