PCI Configuration Process

[bloodline]

Hi all,

I'm slowly adding PCI support to my OS, as I want to use USB (and be free of the antiquated PS2 ports!). Anyway, my PCI driver can enumerate the PCI busses, identifiying all the devices/functions along the way using the x86 IOPorts 0xcf8 and 0xcfc. But I am curious about what I can actually do with the information returned.

Several emulators I have tried have something with a VendorID: 0x1234, DeviceID: 0x1111, of class 0x3. I have identified via the magic of Google that this is an emulated CirusLogic 5446 GFX board. Though this would be a good starting point to figuring out what the PCI configuration registers actually mean. I assume I need to start with the BAR registers, now the emulated gfx card has BAR0 set to 0xFD000008 which is very close to the framebuffer address provided by GRUB at boot with address 0xFD000000... So what is BAR0 actually showing me (8bytes/2pixels into my frame buffer?!?!)?

[8infy]

Hi all,

I'm slowly adding PCI support to my OS, as I want to use USB (and be free of the antiquated PS2 ports!). Anyway, my PCI driver can enumerate the PCI busses, identifiying all the devices/functions along the way using the x86 IOPorts 0xcf8 and 0xcfc. But I am curious about what I can actually do with the information returned.

Several emulators I have tried have something with a VendorID: 0x1234, DeviceID: 0x1111, of class 0x3. I have identified via the magic of Google that this is an emulated CirusLogic 5446 GFX board. Though this would be a good starting point to figuring out what the PCI configuration registers actually mean. I assume I need to start with the BAR registers, now the emulated gfx card has BAR0 set to 0xFD000008 which is very close to the framebuffer address provided by GRUB at boot with address 0xFD000000... So what is BAR0 actually showing me (8bytes/2pixels into my frame buffer?!?!)?

I suggest you at least read the PCI page here, it describes everything you need to know about both PCI & registers.
Lower 4 bits of a memory BAR are reserved for information, such as whether memory is prefetchable, as well as whether its 64/32/16 bits.
Also I recommend that you use MMIO PCI (ECAM) instead of polling IO registers.

[bloodline]

Hi all,

I'm slowly adding PCI support to my OS, as I want to use USB (and be free of the antiquated PS2 ports!). Anyway, my PCI driver can enumerate the PCI busses, identifiying all the devices/functions along the way using the x86 IOPorts 0xcf8 and 0xcfc. But I am curious about what I can actually do with the information returned.

Several emulators I have tried have something with a VendorID: 0x1234, DeviceID: 0x1111, of class 0x3. I have identified via the magic of Google that this is an emulated CirusLogic 5446 GFX board. Though this would be a good starting point to figuring out what the PCI configuration registers actually mean. I assume I need to start with the BAR registers, now the emulated gfx card has BAR0 set to 0xFD000008 which is very close to the framebuffer address provided by GRUB at boot with address 0xFD000000... So what is BAR0 actually showing me (8bytes/2pixels into my frame buffer?!?!)?

I suggest you at least read the PCI page here, it describes everything you need to know about both PCI & registers.
Lower 4 bits of a memory BAR are reserved for information, such as whether memory is prefetchable, as well as whether its 64/32/16 bits.

Ok, I'm glad you pointed that out! Now it all makes sense, Sometimes I just need a nudge in the right direction. Many thanks. Yes, BAR0 just points to the framebuffer and is apparently prefetchable.

Also I recommend that you use MMIO PCI (ECAM) instead of polling IO registers.

Well, I don't have ACPI yet... So I think I'm stuck with the old x86 IO port method for now. But I would hope to move to MMIO as soon as possible, but until then I can work on getting PCI drivers working

[nexos]

As 8infy said, the low 4 bits need to be masked off. There are various other complexities when reading BARs, meaning that you should read the PCI page, or, even better, buy the book PCI systems architecture. Or pay $1000 dollars to PCI-SIG to get the specs .

[Octocontrabass]

Several emulators I have tried have something with a VendorID: 0x1234, DeviceID: 0x1111, of class 0x3. I have identified via the magic of Google that this is an emulated CirusLogic 5446 GFX board.

Your Google doesn't have enough magic. That's a Bochs graphics adapter. The Cirrus adapter is 1013:00B8.

[bloodline]

Several emulators I have tried have something with a VendorID: 0x1234, DeviceID: 0x1111, of class 0x3. I have identified via the magic of Google that this is an emulated CirusLogic 5446 GFX board.

Your Google doesn't have enough magic. That's a Bochs graphics adapter. The Cirrus adapter is 1013:00B8.

As usual, I'm glad I posted here first, cheers guys!

[Ethin]

You can do a lot with the information provides you. Essentially, the PCI configuration process goes something like this:

1. PCI device enumerator scans all PCI devices (including those behind bridges and such).
2. A driver (say, a GFX adapter driver) loads and wants to access an intel GPU in particular. Lets assume that this intel GPU is a skylake GPU. The driver would do the following:
   1. The driver would begin scanning each discovered PCI device in the system.
   2. The driver would check the base class code (BCC), sub-class code (SCC), and program interface (PI) to determine the type of device. In this instance, for the BCC the driver would check for 03h or 04h, meaning a display controller or multimedia device, respectively. For the SCC, the driver would check for either 00h or 80h, meaning a VGA-compatible device or other multimedia device, respectively. And for the PI, the driver would check for 00h.
   3. The driver could perform other checks, such as scanning the vendor and device IDs as well.
   4. Once the driver has determined that it can handle the device in question, it would scan the BARs and map them. It might also enable bus mastering, interrupts, etc.
3. Now that the driver has determined it can handle the device and has performed any other necessary PCI configuration steps that it needs, it can access the devices registers and they will be automatically translated into PCI configuration accesses.

So, in sum, any driver can just:

1. Enumerate PCI devices
2. Scan the base class code, sub-class code, and program interface, if applicable
3. Scan other PCI device properties if applicable
4. Perform any PCI configuration required
5. Map the BAR ranges
6. Ready to drive the device

HTH

[Octocontrabass]

Isn't that a bit backwards?

Usually you would enumerate devices, then use the vendor:device ID to look up and load compatible drivers, then if you don't find a driver that way use the class:subclass:IF to look up and load compatible drivers. I don't see why you would load a driver without knowing which device you want it to drive.

[bloodline]

You can do a lot with the information provides you. Essentially, the PCI configuration process goes something like this:

1. PCI device enumerator scans all PCI devices (including those behind bridges and such).
2. A driver (say, a GFX adapter driver) loads and wants to access an intel GPU in particular. Lets assume that this intel GPU is a skylake GPU. The driver would do the following:
   1. The driver would begin scanning each discovered PCI device in the system.
   2. The driver would check the base class code (BCC), sub-class code (SCC), and program interface (PI) to determine the type of device. In this instance, for the BCC the driver would check for 03h or 04h, meaning a display controller or multimedia device, respectively. For the SCC, the driver would check for either 00h or 80h, meaning a VGA-compatible device or other multimedia device, respectively. And for the PI, the driver would check for 00h.
   3. The driver could perform other checks, such as scanning the vendor and device IDs as well.
   4. Once the driver has determined that it can handle the device in question, it would scan the BARs and map them. It might also enable bus mastering, interrupts, etc.

Fantastic summary! I appreciate you taking the time to bullet point it for me.
I’m still fuzz as to how exactly the BARs work. As my PCI driver builds a database of what it finds on the PCI BUS, I notice that some devices have their BAR registers filled with values, some just have zeros. Am I able to just use whatever value they have set, or should I write my own values there, I assume the documentation for the devices will explain what each BAR points to!?

[*]Now that the driver has determined it can handle the device and has performed any other necessary PCI configuration steps that it needs, it can access the devices registers and they will be automatically translated into PCI configuration accesses.[/list]
So, in sum, any driver can just:

1. Enumerate PCI devices
2. Scan the base class code, sub-class code, and program interface, if applicable
3. Scan other PCI device properties if applicable
4. Perform any PCI configuration required
5. Map the BAR ranges
6. Ready to drive the device

HTH

[nexos]

Not every PCI device has filled BARs. it normally will be up to each driver which BARs should be filled or not. The PCI driver should only hand out BARs, not use them.

Also, device docs should contains which BARs they use (in theory).

[Octocontrabass]

I’m still fuzz as to how exactly the BARs work. As my PCI driver builds a database of what it finds on the PCI BUS, I notice that some devices have their BAR registers filled with values, some just have zeros. Am I able to just use whatever value they have set, or should I write my own values there, I assume the documentation for the devices will explain what each BAR points to!?

During boot, PC firmware enumerates PCI and assigns reasonable default values to the BARs. You can use those values. You don't need to write your own values unless you're working with PCI hotplug or certain non-PC hardware.

The type (memory or I/O) and amount of the resources assigned to each BAR are fixed in hardware. For example, a device that needs 1 MiB of MMIO will have a BAR that only allows memory addresses aligned to 1 MiB boundaries. This is how the firmware (and you) can assign resources without knowing anything about the device.

When you're writing drivers for PCI devices, the documentation for each device should explain the purpose of that device's BARs.

[Ethin]

Isn't that a bit backwards?

Usually you would enumerate devices, then use the vendor:device ID to look up and load compatible drivers, then if you don't find a driver that way use the class:subclass:IF to look up and load compatible drivers. I don't see why you would load a driver without knowing which device you want it to drive.

I mean, that's another way to do it, but either way works I imagine.

[nullplan]

Isn't that a bit backwards?

Usually you would enumerate devices, then use the vendor:device ID to look up and load compatible drivers, then if you don't find a driver that way use the class:subclass:IF to look up and load compatible drivers. I don't see why you would load a driver without knowing which device you want it to drive.

This requires loadable drivers. I think it is reasonable to not start out with those. But yes, it is reasonable to have the PCI enumerator find the correct driver, rather than having the driver find the PCI device.

[nexos]

I have a pretty sophisticated plan for my driver manager. Basically, it will start by loading the ACPI driver to parse the motherboard's basic hardware. The ACPI driver will report devices to the driver managers, which will then try to find a driver and load the driver for it. The ACPI driver will also report buses. Once the ACPI driver finishes enumerating, the driver manager will load and run each bus driver for every bus on the system. This will result in the PCI bus finding devices, and telling the driver manager about them, along with a hint as to how it should load up the driver (i.e, vendorID:classID, or class:subclass:progIF). It will do the same for buses on the PCI bus (i.e, USB, AHCI, SCSI). Any devices the driver manager couldn't find a driver for will be put in a list, and then the user will be told about them at logon.

[bloodline]

I’m still fuzz as to how exactly the BARs work. As my PCI driver builds a database of what it finds on the PCI BUS, I notice that some devices have their BAR registers filled with values, some just have zeros. Am I able to just use whatever value they have set, or should I write my own values there, I assume the documentation for the devices will explain what each BAR points to!?

During boot, PC firmware enumerates PCI and assigns reasonable default values to the BARs. You can use those values. You don't need to write your own values unless you're working with PCI hotplug or certain non-PC hardware.

The type (memory or I/O) and amount of the resources assigned to each BAR are fixed in hardware. For example, a device that needs 1 MiB of MMIO will have a BAR that only allows memory addresses aligned to 1 MiB boundaries. This is how the firmware (and you) can assign resources without knowing anything about the device.

When you're writing drivers for PCI devices, the documentation for each device should explain the purpose of that device's BARs.

So yeah, I'm happy to let the firmware set the BARs to whatever works.

During PCI Bus Enumeration I can find the gfx board which Qemu emulates, and I'm going to use that to test writing PCI based driver code. I've found the Tech Doc for the QEmu graphics chip (the Cirrus Logic GD5446), http://www.vgamuseum.info/index.php/cpu ... 42814695bc

On page 67, this helpfully (though vaguely) lists all the available hardware regs, but I'm confused as to how to access them. The table give an I/O Port address, some of which appear to match up with the PCI Configuration register address/offsets. I'm having trouble picturing how this works as I'm used to hardware registers just sitting in the normal address space.

-Edit- I might have jumped the gun a bit here... So BAR0 is the framebuffer address. BAR1 is 0, but BAR2 is an address 32megs above the framebuffer... Is it possible the hardware registers are mapped here?