Hi,
gusc wrote:1. I couldn't find whether PCI specs say something about Hot-plugging, so I'm wondering can the PCI topography change at run-time?
There's a separate specification called "PCI Hot-Plug Specification" that contains part of the information for hot-plug PCI. It doesn't give any details for the "hot-plug controller" (I assume controller itself isn't standardised and different systems can do different things) or the "hot-plug system driver" (for 80x86 I assume this is mostly done via. ACPI).
gusc wrote:2. What's the best practice to look up a specific device based on it's class and subclass? Do you cache device addresses and class/subclass information or do you re-enumeration at each request?
I normally build a full device tree of hardware, starting with "computer" at the top; then NUMA nodes as children to "computer"; then things like CPUs, memory and PCI host controllers as children to the NUMA node/s. This tree is meant to include everything, including devices that aren't PCI devices (monitors, USB hubs and USB devices, hard disks, ISA devices, etc).
For example; for each piece of hardware in the device tree; I'd keep track of if things like:
- device identification information (e.g. vendorID and deviceID for PCI; EISA ID for ISA, etc), including a human readable text string (e.g. "generic PS/2 mouse" or "LG Flatron L246WHX monitor" or whatever) and (where possible) some sort of UUID or serial number
- if there's a device driver and what state it's in (not started, starting, running, crashed)
- some sort of reference to the device driver (process ID)
- what "power management state" the device is meant to be in (running, reduced power, sleeping, off). Note: if a device doesn't support any power management at all, then the device driver just pretends that does
- which resources (IRQ, IO ports, memory mapped areas) the device uses
- if the device has been assigned to a "user IO group", which group. Note: this is multi-user support. E.g. if there's 3 monitors and 2 keyboards then 2 monitors and one keyboard might be assigned to "user IO group #1", and the other monitor and keyboard might be assigned to "user IO group #2".
- error logging; where an error report might be "device driver crashed", or "controller timeout" or "CRC error" or whatever
- statistics for hardware maintenance and failure prediction. For example, for a mouse you might track how many meters it's been moved and how much time it's been in operation; and then you might setup rules like "needs to be cleaned every 500 meters" and "mean time between failures is 4 years", so that the system can automatically notify the maintenance department when the mouse is due for cleaning or replacing. Note: This is mostly intended for larger companies/businesses/LANs
- physical location. E.g. a printer might be "in corridor outside room #12". This is mostly just notes so maintenance people can find it easily.
There would also be a piece of software called the "device manager" that manages this information and manages device drivers. For example; if a USB controller driver crashes then the device manager would tell any drivers for USB devices that they won't be able to talk to the USB controller; or if there's been no activity for a "user IO group" for a while the device manager might tell device drivers for all devices assigned to that "user IO group" (but not others) to go into a power saving state.
Some of the information would be loaded from disk at boot (e.g. resources for ISA devices, error log, statistics, physical location) and saved to disk if it changes; and some of the information would be entered by humans (e.g. resources for ISA devices, physical location). The device manager would be responsible for that too.
Mostly what I'm trying to say is that PCI devices are a small part of a larger problem; and once you've got software in place to handle the larger problem (e.g. including ISA devices, USB devices, etc) it might not make sense to scan the PCI buses instead of caching the information for PCI devices.
gusc wrote:3. Is the approach mentioned in wiki (
Recursive enumeration) 100% bulletproof? Is it safe to assume that the device 0:0 (bus:device) will always be PCI-to-PCI bus if there's any at all, not, for example, 0:1?
As far as I know; the general idea behind the recursive enumeration is bulletproof; but if bus 0 device 0 is multi-function, then I'd check all functions of the multifunction device for host controllers (just in case).
However; it won't work for hot-plug PCI when devices are inserted after boot (as the hardware wouldn't have existed during boot when the firmware assigned bus numbers and/or resources to PCI devices). This means that if you intend to support hot-plug PCI then you'll need to be able to do the "Recursive Scan With Bus Configuration" thing. Note that you may also need to be able to reconfigure existing device's resources (possibly while the devices are being used). For example, if a bridge is configured to forward physical memory accesses in the range 0xD0000000 to 0xD8000000 to its secondary bus, and then someone (using hot-plug PCI) inserts 3 video cards (with 512 MiB of memory mapped IO each) onto that secondary bus; then you may have to reconfigure other bridges (and all devices connected to the other bridges) to create enough space; then reconfigure the bridge that the video cards got plugged into and any existing devices on that bridge; just so that you've got enough space for the inserted video cards and can start configuring them.
Note: This "reconfigure resources while a device is running" should sound scary. With careful design, it should be possible to be able to change a device's resources without the device driver knowing or caring - e.g. "freeze" the driver, then change the physical address of the device's memory mapped IO areas, then change the device driver's page tables to suit, then "unfreeze" the driver. You may need to rearrange IO ports too, and it's possible to come up with a scheme to allow the IO ports a device uses to be changed without the driver knowing/caring (e.g. make the driver use a kernel function to access IO ports; or emulate instructions like "IN" and "OUT" in the general protection fault handler; such that IO ports a driver uses are virtualised).
Cheers,
Brendan
