OSDev.org

hi all, I have heard you must get into ACPI but that includes stuff i don't need and the ACPI driver would be very hard to write, so my question is how to start more CPUs in the least amount of code

thx

Hi,

GLneo wrote:hi all, I have heard you must get into ACPI but that includes stuff i don't need and the ACPI driver would be very hard to write, so my question is how to start more CPUs in the least amount of code

To use ACPI all you really need to do is parse some tables to find the APIC IDs of the extra CPUs and a little more information. You don't need any power management or run-time stuff.

Alternatively you could use Intel's Multiprocessor Specification tables. This is fairly similar to using ACPI - find and parse some tables to get the APIC IDs of CPUs and a little more information

Lastly you can broadcast the initialisation sequence to all CPUs without detecting anything. In this case even broken CPUs that the BIOS disabled will respond (or try to) and people who have disabled hypertheading in the BIOS will still get hyperthreading. You won't even know if there's any CPUs to start or if all of them actually did start when they were meant to. You also won't know where the local APICs are (although most of the time they're at the default address).

There's also a little more information that is available via. ACPI or the Multiprocessor Specification tables. This is mostly information on how to disable the PICs and how the I/O APICs are configured.

Parsing the ACPI tables and/or Multiprocessor Specification tables is probably a lot easier than you think. You could write code that works on some SMP computers that doesn't parse either of these tables, but IMHO if you can't figure out how to parse these tables then you might not be ready for SMP yet...


Cheers,

Brendan

well I could read the table ( mabey ) but I don't understand where these tables are? also how do I send the startup to all CPUs?


thx!

Are the intel multiprocessor specs and ACPI multiprocessor tables incompatible?

if you are looking for code: Note, it conforms to all of brendan's "points of bad code"

Hi,

Tyler wrote:Are the intel multiprocessor specs and ACPI multiprocessor tables incompatible?

Intel's Multiprocessor Specification (or MPS for short) is a set of tables that contain information needed to use SMP (and nothing else). ACPI is a set of tables that contain the same information needed to use SMP (and a lot of other stuff we can ignore for the purpose of this discussion).

MPS is older and easier to understand because it doesn't contain other stuff, gives better descriptions of what's going on and also contains more information (like the algorithm used to actually start the CPUs). ACPI is newer and is meant to make MPS obsolete (although I haven't found an SMP system with ACPI that doesn't also have MPS tables for backwards compatability). ACPI also includes things that are useful once you move beyond plain SMP (i.e. more tables that describe NUMA machines).

GLneo wrote:well I could read the table ( mabey ) but I don't understand where these tables are? also how do I send the startup to all CPUs?

For the reasons mentioned above, I'd implement support for MPS first, and worry about ACPI after you've got that working. Most OS's will check for ACPI tables and use them, but will also use MPS if ACPI isn't found.

I'd start by downloading and reading Intel's Multiprocessor Specification.

To make this easier I'll provide a description of how I do things and links to the source code...

Note: My code to find the MPS and ACPI tables is seperate from the code that uses these tables. This isn't necessary and is a result of my modular OS design (I have a Boot Manager that has an API that allows other modules to ask for the addresses of MPS tables and ACPI tables).

There's several steps that are mostly the same for both MPS and ACPI, so I'll describe them in parallel.


Step 1 - Find The Table/s

For both ACPI and MPS, the tables live somewhere, and you need to search for them. This mainly involves scanning different areas (the EBDA and the BIOS) for an identifier ("_MP_" and "RSD ") and checking a checksum.

• My code to find the MPS tables
  My code to find the ACPI tables

Step 2 - Find Relevant Entries In The Table/s

This is slightly different for ACPI and MPS. For MPS each table has a length, and the tables follow each other in memory. Each table has a unique numeric identifier. To find a specific table you'd start from the beginning and skip tables until you find the right one. For ACPI each table has a 4 character ASCII identifier, and there's a "Root Directory" that contains pointers to each table, so you scan this directory until you find the right pointer. My Boot Manger's API also includes functions to determine how many tables there are with a certain identifier (and allows other modules to ask for the Nth table of a certain type).

• My code to find a specific MPS table
  My code to find a specific ACPI table

Step 3 - Parse The Table/s

For MPS there's "default configurations" which a describe a specific configuration (always 2 CPUs where one uses APIC ID 0x00 and the other uses APIC ID 0x01), so first we check if MPS is using a default configuration and use assumed values (instead of parsing tables) if it is. Most machines don't use a default configuration. For most machines there's a table for each "local APIC" (or CPU), so my code asks the Boot Manager how many of these there are, then does a loop that gets each table and parses it. For ACPI there's a single table that describes all APICs (local APICs and IO APICs). My code gets the address of this table and scans through it, ignoring anything that doesn't describe a local APIC.

• My code to find a specific MPS table
  My code to find a specific ACPI table

Step 4 - Start Each CPU

For the previous step, each time the code found a local APIC (or a CPU) in the tables it created a structure containing initial information about that CPU. In this step we go through the list of "CPU data structures" and start each CPU, except for the BSP CPU (which is already running). This involves sending some IPIs (Interprocessor Interrupts) from the BSP CPU to the "sleeping" CPU to wake it up. The sequence of IPIs (and the timing requirements) are described in Intel's Multiprocessor Specification. To be honest I don't follow Intel's algorithm here (I found the delays are too short to accurately measure) and did my own slightly different thing. Also, starting CPUs with older local APICs is slightly different. Intel's algorithm describes how they are started, while my OS refuses to use SMP in this case (I don't support SMP for 80486 or older computers, and they'd boot as single CPU machines).

When a CPU is started it starts in real mode at the address you tell it (which must be below 1 MB with certain aligment restrictions). My code puts a JMP instruction at a suitable address and tells AP CPUs to start there. My boot code also runs in real mode so I don't switch the AP CPUs to protected mode at this stage (but it does set GS to "flat mode").

• My code to start detected CPUs

Conclusions

The way I do things is a little more complicated than necessary (due to the modular nature of my OS). It'd be possible to have a single routine that parses the ACPI or MPS tables and starts CPUs as it finds them.

I have skipped some details in my descriptions above (like full descriptions of tables, etc), and my source code might be a little hard to follow without these details. This is deliberate - you'll learn about these details (and be able to follow my code) after you've read Intel's Multiprocessor Specification and the relevant part of the ACPI Specification.


Cheers,

Brendan

Hi,

Combuster wrote:Note, it conforms to all of brendan's "points of bad code"

Hehe, indeed..

It does remind me of one thing though - for some CPUs (Pentium IIRC) the local APICs are a little broken and you need to do a dummy read from the local APIC before doing any write to the local APIC.

Although this doesn't explain why you're doing the "MOV EAX, [LAPIC_REG_ICR_HI]" immediately before doing "MOV EAX, [LAPIC_REG_ICR_LOW]", and I can't think of a reason for the first read at all (it looks like a dummy read that doesn't precede a write).

I'm also wondering why you disable caches?

As far as I can tell, you could remove 8 instructions from this code without making it more crude...


Cheers,

Brendan

Brendan wrote:I'm also wondering why you disable caches?

Big book of patterns wrote:The 4-KByte APIC register address space should be mapped as uncacheable (UC), refer to
Section 9, “Memory Cache Control”, in Chapter 9, Memory Cache Control.

Writes must go to uncached memory, and since I don't have paging when i'm booting APs, CR0.CD is the most obvious way.

about the dummy read, in a previous version i always loaded ICR_HI and never bothered checking wether the read was still necessary after optimising it out.

btw, who cares about two extra memory accesses in the face of a timer loop?

Hi,

Combuster wrote:

Brendan wrote:I'm also wondering why you disable caches?

Big book of patterns wrote:The 4-KByte APIC register address space should be mapped as uncacheable (UC), refer to
Section 9, “Memory Cache Control”, in Chapter 9, Memory Cache Control.

Writes must go to uncached memory, and since I don't have paging when i'm booting APs, CR0.CD is the most obvious way.

The most obvious way is to assume the BIOS configured the MTRRs correctly, and that this area is already uncacheable (and remains uncacheable regardless of what you do with CR0 and paging)...

Combuster wrote:about the dummy read, in a previous version i always loaded ICR_HI and never bothered checking wether the read was still necessary after optimising it out.

btw, who cares about two extra memory accesses in the face of a timer loop?

People maintaining the code, who might see you're doing something with "LAPIC_REG_ICR_HI" and wonder if the destination field was meant to be set or not, or if this read is there for a reason (like the dummy reads for broken local APICS)...


Cheers,

Brendan

The most obvious way is to assume the BIOS configured the MTRRs correctly, and that this area is already uncacheable (and remains uncacheable regardless of what you do with CR0 and paging)...

MTRRs and the Pentium I?

A quick round in the manuals didnt reveal any magic that occurs within P1s that make the APIC range automatically uncacheable. Maybe you can enlighten me on that one.

Hi,

Combuster wrote:

The most obvious way is to assume the BIOS configured the MTRRs correctly, and that this area is already uncacheable (and remains uncacheable regardless of what you do with CR0 and paging)...

MTRRs and the Pentium I?

A quick round in the manuals didnt reveal any magic that occurs within P1s that make the APIC range automatically uncacheable. Maybe you can enlighten me on that one.

It's in a document called "Pentium(R) Processor Family Developer's Manual" (a file named 24142805.pdf), which is intended for hardware developers (not software developers).

For older CPUs the chipset itself if responsible for telling the CPU if each memory access is cacheable or not via. a signal called "#KEN" (or "Cache Enable").

As far as I can tell, (for read accesses) the CPU sends an address on the bus and the memory controller either asserts or deasserts #KEN. If #KEN was asserted the CPU expects a cache line fill. If #KEN wasn't asserted the CPU expects a single transfer read (a byte, word or dword I guess).

Of course I'm not a hardware developer, and I didn't bother reading the full document (or figuring out how writes work).

I would assume that the idea of the MTTRs was to shift this into the CPU, so that the CPU doesn't need to wait for the chipset to assert or deassert #KEN during transfers.

Of course it would be possible to shift the local APIC's base address to something silly (i.e. map the local APIC to 0x00000000). In this case you can expect problems with cacheing and the programmer would need to ensure that cacheing is disabled.


Cheers,

Brendan