Discovering your system

[jal]

Say I have a modern system, one I buy in a store today, either x86 or x86-64. I'm writing an OS to run on that system, and I want to discover everything there's to know about its capabilities, so the OS can run. First up would probably be memory detection, but what's next, and in which order? Probably do a CPUID, then check the ACPI tables? Are there any more data structures to parse (non-depricated ones, that is)? Does ACPI give all device information needed (e.g. busses, so I can query the PCI(e) etc.?). There doesn't seem to be much in the Wiki, at least not in a coherent format, and most tutorials, bare bones etc. run on a plain 386 and approach the hardware like that (i.e. writing directly to PCI ports, assuming there's a PIT and PIC, etc.).

Next, once the OS has gathered all this information, the OS is probably going to initialize some stuff, like putting the (L)APICs into use, switch to long mode on a 64-bit machine, etc. No doubt there's a sane order in which to do these things, based on the information gathered.

What I'm at is putting up some decent information on the Wiki shortly describing all this stuff in steps, so someone with outdated knowledge (like me, I'm from the 90s (code-wise, that is)) can quickly get a grasp of what to do.


JAL

[geppy]

If you are an asm programmer then I would suggest doing as minimum as possible until you are in long mode. Working with 16 registers is a joy.
If you write in C then it depends on your compiler but compiler is unlikely to utilize all regs so your code will larger and slower(as long as you choose same algorithms)

IMHO:
Slice 1st 4gb of ram in protected mode. In long mode you'll have to deal with paging, will be slower. So you'll need separate code for long mode to slice further. (my phys mem manager uses unused & unpaged ram for data structures)
ACPI parsing along with Multi CPU setup in long mode: parse ACPI, setup lapic of current cpu, setup timer interrups(rtc,lapic timer, hpet), bring online other cpus. While you wait for CPUs to initialize you deal with device enumeration or something else.
In real mode you can ask BIOS how many PCI busses there is and determine regions of available ram. Maybe determine VESA modes and switch if you plan to display some info during bootup which really should last less than 1sec.

[bewing]

IMO, there are two completely different issues here. The sane order for development of the hardware init sequence, and the sane order of the init sequence in the "final" OS.

I would say that the sane order for the final OS is almost irrelevant, since it is always many years away from your current development stage.

A sane order for hardware init development keeps getting interrupted by building/debugging the "real" (non-hardware) parts of the OS -- the executives, the API, etc.

But a sane hardware sequence might start something like:
Assume a fixed number of contiguous megs of physical ram
Assume one monitor, one keyboard, one PS2 mouse
Assume a PIC/PIT
Build a textmode VGA driver
Build a keyboard driver
Assume one floppy or PIO PATA hard disk
Build a pmode floppy/harddisk driver
Build a PCI bus scanner
Build a physmem manager
After you get all that debugged, then you probably should set up multitasking -- because the rest of the hardware init stuff works much better/faster when it's multitasked. Or you can do the multitasking FIRST, so all your drivers are thread-safe.

And then you build your generic "driver stack" for USB and TCP/IP.
And then you go back and replace all those "assumptions" with better code that makes fewer assumptions, in whichever order you feel like. (And replace your drivers with more capable ones.)

As for your question about what other hardware info you need to look for in memory, there is the PlugNPlay data scattered through memory.

(PS. and you were the one who specifically requested a Linux version of the bochs GUI, so I hope you have volunteered to test it out!)

[Brendan]

Hi,

I'm writing an OS to run on that system, and I want to discover everything there's to know about its capabilities, so the OS can run. First up would probably be memory detection, but what's next, and in which order?

My intended order of detection goes like this:

Boot Code

• Physical address space map
• Get disk drive information (e.g. int 0x13, AH=0x48 or int 0x13, AH=0x08). Mostly so the OS can figure out which devices correspond to which BIOS drive numbers for OS (re)installation.
• Get some SMBIOS information (mostly only motherboard vendor and brand strings)
• Get some PIC IRQ routing information (PCI IRQ Routing Table Specification)
• Find ACPI tables and/or MP specification tables
• CPUs (how many, topology, features)
• Physical address space topology (which NUMA domains areas belong to)

After Kernel Started

• Get PCI host controller vendor ID and product ID
• Start motherboard driver (found using combination of SMBIOS motherboard vendor and brand strings, and PCI host controller vendor ID and product ID)
• Scan PCI bus and disable all PCI devices
• Do "ISA Plug & Play" scan and disable all "ISA Plug & Play" devices (unless ISA is disabled by motherboard driver)
• Probe for any plain ISA devices (unless ISA is disabled by motherboard driver)
• Get I/O APIC or PIC IRQ routing information
• Enable and assign resources to all "ISA Plug & Play" devices (unless ISA is disabled by motherboard driver)
• Enable and assign resources to all PCI devices

After that, it's ready to start device drivers (in any order). However, this is a recursive thing. For example, if a device driver for a USB controller becomes ready then the OS will start scanning for any devices attached to the USB controller and find more devices.

I also plan on implementing it in exactly the same order, however the order I write device drivers and pieces of code to probe for old ISA devices will be mostly unpredictable (although I normally start with a keyboard driver and a "legacy framebuffer" video driver).


Cheers,

Brendan

[jal]

(PS. and you were the one who specifically requested a Linux version of the bochs GUI, so I hope you have volunteered to test it out!)

Not yet, as I have very little time for OSdeving currently (wife, kids, work, holiday season, etc.), but I will try it out as soon as I can.


JAL

[jal]

My intended order of detection goes like this:

Thanks, that is quite informative. If one is assuming legacy-free operation, is ISA detection needed at all?


JAL

[Love4Boobies]

Sorry if this is slightly off-topic maybe. But it's about discovery nontheless. Is SMBIOS meant to be a replacement for PnP BIOS? Is PnP BIOS being deprecated?

[Brendan]

Hi,

If one is assuming legacy-free operation, is ISA detection needed at all?

That depends a little on who said "legacy free". For example, for thin clients "legacy free" often means they didn't put PS/2 ports on the case and you have to use USB keyboard and mouse (but all the legacy stuff is still there, including A20, BIOS, PS/2 controller chip, PIC, etc).

However, for ISA detection there's 2 things to consider - ISA cards plugged into the motherboard, and legacy/ISA devices built into the chipset. It's possible that "legacy free" also implies "no ISA slots" (it's hard to tell with modern computers, because most of them don't have ISA slots anyway), but then you still have to worry legacy/ISA devices built into the chipset.

Sorry if this is slightly off-topic maybe. But it's about discovery nontheless. Is SMBIOS meant to be a replacement for PnP BIOS? Is PnP BIOS being deprecated?

SMBIOS is meant to be used by utilities that help with keeping track of what hardware a company owns (for accountancy and hardware maintenance/upgrade purposes - for example, see the Wikipedia page on Fixed Assets Management), and was never meant to be a replacement for anything (except earlier versions of itself).

The PnP BIOS Specification is (IMHO) mostly a software interface to the features introduced by the PnP ISA Specification. However, the PnP BIOS Specification (Version 1.0A) was published in May 1994, and is the only version of this specification I've found. PCI 1.0 was released in June 1992 and PCI 2.0 was released in April 1993; and PCI is a far better specification (better bus, better support for configuration, etc). Because of this most expansion card manufacturers didn't bother with ISA PnP at all (they just went straight to PCI). This means that almost all ISA cards don't support the ISA PnP specification; which also means that the PnP BIOS can't get any information from almost all ISA cards.

Of course an OS can't rely on PnP BIOS being supported either (computers with Pentium and older CPUs existed before the PnP BIOS specification was released). However, it is possible for an OS to support the ISA PnP specification and use this to get information from (and configure) ISA cards that support the PnP ISA specification, even though the BIOS itself doesn't support PnP.

The PnP BIOS specification might be useful for detecting legacy/ISA devices built into the chipset (if you can't find other ways, e.g. ACPI), but you could probably close your eyes and guess and be fairly accurate anyway (which makes manual probing relatively easy).

Basically, IMHO the PnP BIOS was crappy when it was released, and since about 1996 to 2000 it started being replaced by ACPI (which is also crappy, for entirely different reasons).

Of course I should mention that in general I'm "overly skeptical" - I won't assume any BIOS is capable of doing anything correctly if I've got a choice.


Cheers,

Brendan

[quok]

If one is assuming legacy-free operation, is ISA detection needed at all?

That depends a little on who said "legacy free". For example, for thin clients "legacy free" often means they didn't put PS/2 ports on the case and you have to use USB keyboard and mouse (but all the legacy stuff is still there, including A20, BIOS, PS/2 controller chip, PIC, etc).

However, for ISA detection there's 2 things to consider - ISA cards plugged into the motherboard, and legacy/ISA devices built into the chipset. It's possible that "legacy free" also implies "no ISA slots" (it's hard to tell with modern computers, because most of them don't have ISA slots anyway), but then you still have to worry legacy/ISA devices built into the chipset.

I was reading the X58 Chipset datasheet last night and saw that the X58 supports all these "legacy" devices on an ISA bus, but also has options to disable the functionality entirely. For example, the "legacy" VGA framebuffer area can be disabled entirely and turned in to usable RAM, or it can be forwarded to a detected display adapter in the system. Then the display adapter itself (in the case of i945 based systems, anyway) can decide to map the "legacy" framebuffer to the beginning of its real framebuffer address or to implement it as a separate plane on the card.

Likewise, the traditional PCI I/O ports are considered as Intel by "legacy" since PCI-e 2.0 is out, and the X58 has options in it to respond to the legacy ports or not.

I won't even get in to the options the X58 has to emulate "legacy" FSB on the new QPI interface.

Basically my point is that even the same chipset may end up reporting different devices depending on how the BIOS set it up, so relying on all chipsets to have all the same devices is a bad idea. Oh, and as Brendan points out, what is considered legacy depends entirely on who is talking.

[Owen]

All modern motherboards have an ISA bus on them. Or rather, they have an LPC (Low Pin Count) bus, which is a cheapified version of ISA: 6 pins (IIRC), runs at the 33MhZ PCI clock. It works out slightly faster than ISA when you account for the fact that it has to multiplex the address and data on those 8 pins.

It's completely the same as ISA to software. And it is used to connect a sum total of two or three devices:

• SuperIO chipset with legacy devices (Excluding the VGA which is implemented on the chipset or graphics card)
• BIOS ROM
• If you have one, (AKA If your a Matcel owner) the TPM

It doesn't support ISA P&P.