OSDev.org

Hi,

I've some confusion about the linux kernel boot options iommu=on and iommu=pt in the context of virtualization environments (e.g. QEMU/KVM). From my understanding iommu=on turns on the IOMMU device unit (if it exists in HW of course) for all uses. Adding further the option iommu=pt does mean the following:

• from Host OS point of view the IOMMU unit is actually disabled (DMA target addresses are the same as host system physical addresses)
  
  physical devices (e.g. NIC) configured in passthrough (e.g. NIC PCI passthrough) assigned to a Guest running in a VM, continue to use the IOMMU unit (their DMA target addresses are actually I/O virtual addresses assigned to the DMA-capable passthrough NIC by the Guest and translated by IOMMU DMARs to access memory)

Does it makes sense ? Thank you.

ps. from the Intel manual 'vt-directed-io-spec - rev 3.3' section 10.4.3 the PT bit into Extended Capability Register is set if HW does support IOMMU passthrough. For each PCI device identified by its B/D/F the looked up Context Entry (or Scalable-Mode PASID Table Entry) specifies the Translation Type in TT (or GPTT) field (see section 9.3 - 9.6). Pass-through mode (TT = 10b or GPTT = 100b) actually means IOMMU does not perform any form of DMA target address translation -- i.e. DMA target addresses are actually system physical addresses.

Any feedback ? Thank you.

Please be patient. We are not being paid to answer your questions and we answer them in our own time.

PCI passthrough requires an IOMMU. Without an IOMMU, the hypervisor has to emulate the device.

Accessing devices through an IOMMU adds some overhead. When you're not doing PCI passthrough, you probably don't want to use the IOMMU.

The "iommu=pt" option gives you the best (or at least fastest) of both worlds. The IOMMU is used for PCI passthrough, and it isn't used for anything else.

Octocontrabass wrote:PCI passthrough requires an IOMMU. Without an IOMMU, the hypervisor has to emulate the device.
Accessing devices through an IOMMU adds some overhead. When you're not doing PCI passthrough, you probably don't want to use the IOMMU.

The "iommu=pt" option gives you the best (or at least fastest) of both worlds. The IOMMU is used for PCI passthrough, and it isn't used for anything else.

Thank you for the answer.

AFAIK the IOMMU HW unit is actually employed only from DMA-capable devices to perform DMA accesses either to physical memory (i.e. DRAM) or to other MMIO devices's registers.

So when you say Accessing devices through an IOMMU adds some overhead are you actually referring to DMA accesses (transactions) performed from DMA-capable devices as described in my sentence above ? Thank you.

Yes. When the IOMMU has to walk its page tables to map a DMA transaction to physical memory, the DMA transaction has to wait.