Intel VT-d IOMMU passthrough

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
cianfa72
Member
Member
Posts: 73
Joined: Sat Dec 22, 2012 12:01 pm

Intel VT-d IOMMU passthrough

Post by cianfa72 »

Hi,

I've some confusion about the linux kernel boot options iommu=on and iommu=pt in the context of virtualization environments (e.g. QEMU/KVM). From my understanding iommu=on turns on the IOMMU device unit (if it exists in HW of course) for all uses. Adding further the option iommu=pt does mean the following:
  • from Host OS point of view the IOMMU unit is actually disabled (DMA target addresses are the same as host system physical addresses)

    physical devices (e.g. NIC) configured in passthrough (e.g. NIC PCI passthrough) assigned to a Guest running in a VM, continue to use the IOMMU unit (their DMA target addresses are actually I/O virtual addresses assigned to the DMA-capable passthrough NIC by the Guest and translated by IOMMU DMARs to access memory)
Does it makes sense ? Thank you.

ps. from the Intel manual 'vt-directed-io-spec - rev 3.3' section 10.4.3 the PT bit into Extended Capability Register is set if HW does support IOMMU passthrough. For each PCI device identified by its B/D/F the looked up Context Entry (or Scalable-Mode PASID Table Entry) specifies the Translation Type in TT (or GPTT) field (see section 9.3 - 9.6). Pass-through mode (TT = 10b or GPTT = 100b) actually means IOMMU does not perform any form of DMA target address translation -- i.e. DMA target addresses are actually system physical addresses.
cianfa72
Member
Member
Posts: 73
Joined: Sat Dec 22, 2012 12:01 pm

Re: Intel VT-d IOMMU passthrough

Post by cianfa72 »

Any feedback ? Thank you.
Ethin
Member
Member
Posts: 625
Joined: Sun Jun 23, 2019 5:36 pm
Location: North Dakota, United States

Re: Intel VT-d IOMMU passthrough

Post by Ethin »

Please be patient. We are not being paid to answer your questions and we answer them in our own time.
Octocontrabass
Member
Member
Posts: 5563
Joined: Mon Mar 25, 2013 7:01 pm

Re: Intel VT-d IOMMU passthrough

Post by Octocontrabass »

PCI passthrough requires an IOMMU. Without an IOMMU, the hypervisor has to emulate the device.

Accessing devices through an IOMMU adds some overhead. When you're not doing PCI passthrough, you probably don't want to use the IOMMU.

The "iommu=pt" option gives you the best (or at least fastest) of both worlds. The IOMMU is used for PCI passthrough, and it isn't used for anything else.
cianfa72
Member
Member
Posts: 73
Joined: Sat Dec 22, 2012 12:01 pm

Re: Intel VT-d IOMMU passthrough

Post by cianfa72 »

Octocontrabass wrote:PCI passthrough requires an IOMMU. Without an IOMMU, the hypervisor has to emulate the device.
Accessing devices through an IOMMU adds some overhead. When you're not doing PCI passthrough, you probably don't want to use the IOMMU.

The "iommu=pt" option gives you the best (or at least fastest) of both worlds. The IOMMU is used for PCI passthrough, and it isn't used for anything else.
Thank you for the answer.

AFAIK the IOMMU HW unit is actually employed only from DMA-capable devices to perform DMA accesses either to physical memory (i.e. DRAM) or to other MMIO devices's registers.

So when you say Accessing devices through an IOMMU adds some overhead are you actually referring to DMA accesses (transactions) performed from DMA-capable devices as described in my sentence above ? Thank you.
Octocontrabass
Member
Member
Posts: 5563
Joined: Mon Mar 25, 2013 7:01 pm

Re: Intel VT-d IOMMU passthrough

Post by Octocontrabass »

Yes. When the IOMMU has to walk its page tables to map a DMA transaction to physical memory, the DMA transaction has to wait.
Post Reply