PCI Over Again

[CmpXchg]

Hi!

I know this topic springs up from time to time, but I failed to find an answer to the question I'm about to ask.

So I want to implement a proper PCI enumeration. In the topic PCI Configuration Registers Brendan gave an excellent reply on how to get size of a BAR (of the area it represents, to be exact). However, it's unclear to me what flags are there in a BAR and what they stand for.

Let me quote some of his code:

Code: Select all

  // IO space
...
BAR = PCI_read_32(bus, device, function, BAR_offset);
BAR = BAR & 0xFFFFFFFC;          // Clear flags
...

So the operation BAR = BAR & 0xFFFFFFFC zeroes out two lowest bits of BAR. Hex "C" stands for "1100", right? So there are 2 flags. Ok, I know the LSB indicates if it's a memory region or I/O space area, but what does the other flag stand for?

Let's see further...

Code: Select all

if ( (BAR & 1) == 0) {
            // Memory
            type = (BAR >> 1) & 0x03;
            if( (type & 2) == 0) {
                // 32 bit Memory
                ...
            } else {
                // 64 bit Memory
               ....
...

So, 64-bit Memory? How come? It can exist only on x86-64 platform, right? I must admit this piece of code is a little difficult for me to understand, so please tell me what do the bits stand for, without shift and multiple AND operations.

I'll be very thankful if you make this stuff clear for me.

Please excuse me for asking more than 1 question, but I'm afraid there's something else I'd want to ask.

So I'd love to know more about the "Header type" field in the config space. In the topic of pci enumeration I could read:

Some devices can have multiple functions, which you can identify because their HeaderType byte has the 7th bit set (0x80). For those devices, you have to poll every function.
Some devices can be pci-to-pci bridge (they'll have a (headertype & 0x7F) == 1) In the case of such a bridge, there's a place in the header where you can read the amount of "subordinate" busses (and thus the highest bus to be scanned).

Quite a valuable piece. But doesn't (headertype & 0x7F) == 1) mean that headertype=1? Besides, in Linux sources there's following:

Code: Select all

PCI_HEADER_TYPE_NORMAL		EQU		0
PCI_HEADER_TYPE_BRIDGE		EQU		1
PCI_HEADER_TYPE_CARDBUS		EQU		2

which makes me sure Bridges have a 1 in Headertype. So my question is: how exactly should one interpret Headertype? Are these the only possible values for Headertype? What should I do if come across a CARDBUS?

Once again, sorry for boring questions. I'll appreciate any information on that. Good links are more than welcome. CmpXchg

[Brendan]

Hi,

For BARs, the first bit determines if the BAR is for a range of I/O ports or a range of addresses in the physical address space.

If bit 0 is set, then it's a range of I/O ports. In this case bit 1 is reserved, and the other bits determine the "address" (base I/O port number) and size of the range (more on that later).

If bit 0 is clear, then bits 1 and 2 are the type (00 = 32-bit, 01 = reserved, 10 = 64-bit, 11 = reserved), and bit 3 determines if the area is prefetchable (bit set) or not prefetchable (bit clear). The remaining bits determine the starting address and size of the range.

To determine the size of an area, you set all bits to ones and read it back to see how many of the bits are hardwired to zero. For example, if you read back the value "0xFFFFF000" then you know the range is 4 KB (and that it's a range of addresses not a range of I/O ports, and that it uses 32-bit addressing, and that it's not prefechable).

Once you know what the card needs you'd find somewhere to map the card (allocate resources for it), then set the address by writing to the BAR/s again. Alternatively, you could rely on the BIOS's configuration and just read the address from the BAR (or perhaps read the BAR, then write ones to it, then find out the size of the area, then restore the original values).

Lastly, there's 64-bit addressing. PCI cards aren't "80x86 only" in that the other architectures (e.g. 64-bit RISC) can use the same PCI cards. In addition, "32-bit only" 80x86 CPUs actually support 36-bit addressing (with PSE26 or PAE). For example, it'd be entirely possible to setup a video card to use the (1 GB) range from 0x0000000F00000000 to 0x0000000F3FFFFFFF and then use PAE to access it (without needing a 64-bit CPU).

There are a few problems here though - some bridges are dodgy and don't forward 64-bit accesses correctly, and if you're assigning your own resources (instead of relying on the BIOS to do it right when you know the BIOS won't use 64-bit ranges for backward compatibility reasons) then you also need to make sure that bridges are configured correctly. Basically you can't set an arbitrary address in a device's BAR and assume bridges will forward accesses within that address range to the sub-bus the device is attached to.

Also note that for 80x86 32-bit I/O port numbers don't make sense (the CPU only supports 16-bit I/O port numbers) - just use the lowest 16-bits of the BAR for I/O ports.

Quite a valuable piece. But doesn't (headertype & 0x7F) == 1) mean that headertype=1?

No - IIRC it could mean that headertype = 0x01 (a single-function PCI bridge) *or* headertype = 0x81 (a multi-function PCI bridge).

Are these the only possible values for Headertype?

AFAIK these are the only header types that are defined up until now; however, they could release a new version of the PCI specification tomorrow (or some related specification, like "PCI bridges" or "PCMCIA") that defines more header types.

What should I do if come across a CARDBUS?

Until you start implementing support for CardBus, just disable the device. I'd assume writing zero to the Device Control register would still work, even though it's a different header type. It's probably described in some specification somewhere (possibly the PCMCIA specification) if you want to "do it right" and make sure...


Cheers,

Brendan

[Korona]

Is there any reliable way to find unused i/o addresses that I can assign to a PCI device or should I rely on the base addresses set by the BIOS?

[CmpXchg]

Thank you, Brendan!

Now I feel much more confident in this area, no need to rely on the guesswork etc., etc. By the way, does anyone on this board dare enable 36-bit addressing?

Actually I have one more question about PCI. So - in a book I read that register 00h-0Ch layout is same for every PCI device, but starting from 10h and until 40h register space depends on Header Type. (Device-specific registers go from offset 40h upwards).

Probably the above statement means that for bridges I can read

* Header type 1 (PCI-to-PCI bridges) */
#define PCI_PRIMARY_BUS 0x18 /* Primary bus number */
#define PCI_SECONDARY_BUS 0x19 /* Secondary bus number */
#define PCI_SUBORDINATE_BUS 0x1a /* Highest bus number behind the bridge */

So bridges don't make use of BARs. (The quote comes from the same pci enumeration topic.)

May I ask an awkward question: is there nothing more I can read for bridges? I have a specification for my motherboard, which keeps silent about the bridges on the PCI bus.

Thanks again in advance

@Korona: Well, build an I/O map as accurately as you can. To do it will require you to read BARs, probably ask PnP BIOS, SMBios and if you like it - ACPI. Despite the belief the ACPI is next to impossible to implement, there have been attempts. See ACPI poweroff topic.

Also note that I/O adresses 0x0000-0x0CFF are used by certain legacy devices and cannot be remapped (as far as I am concerned). For example, here's my detailed map of this area, taken from the datasheet:

Code: Select all

Table 5.  System I/O Map 
Port Function  Actual Port Decoding 
00-1F  Master DMA Controller   0000 0000 000x nnnn 
20-3F  Master Interrupt Controller  0000 0000 001x xxxn 
40-5F  Timer / Counter  0000 0000 010x xxnn 
60-6F  Keyboard Controller  0000 0000 0110 xnxn 
(60h)  KBC Data  0000 0000 0110 x0x0 
(61h)  Misc Functions & Spkr Ctrl 0000 0000 0110 xxx1 
(64h)  KBC Command / Status  0000 0000 0110 x1x0 
70-77  RTC/CMOS/NMI-Disable  0000 0000 0111 0nnn 
78-7F  -available for system use-  0000 0000 0111 1xxx 
80  -reserved- (debug port)  0000 0000 1000 0000 
81-8F  DMA Page Registers  0000 0000 1000 nnnn 
90-91  -available for system use-  0000 0000 1001 000x 
92  System Control  0000 0000 1001 0010 
93-9F  -available for system use-  0000 0000 1001 nnnn 
A0-BF  Slave Interrupt Controller  0000 0000 101x xxxn 
C0-DF  Slave DMA Controller  0000 0000 110n nnnx 
E0-FF  -available for system use-  0000 0000 111x xxxx 
100-CF7  -available for system use* 
CF8-CFB  PCI Configuration Address  0000 1100 1111 10xx 
CFC-CFF  PCI Configuration Data  0000 1100 1111 11xx 
D00-FFFF  -available for system use-

As you can see, there are small gaps available for mapping, but I think such little areas would suffice no device.

Edit: Well, the area of 0x0100-0x0CF7 seems large enough for some devices, but it's still best not to remap anything here.

Hope this gives you a hint on how to accomplish the task,
CmpXchg