OSDev.org

[EDIT: my main questions have been answered, the rest is more about curiosity]

Hi there !


The short version (tl;dr):
-- Working on an "Intel Core i7 (Ivy Bridge)" CPU
-> Is there a way to know if an AP core of a multi-core CPU has been enabled ?
-> Is there a way to switch from multi-core to mono-core processing at OS startup? (shutdown/disable AP cores ?)


The story:
I've been interested in trying to build some kind of "RTOS" since a while, (I only have an Intel Core i7 CPU to try that out !)

What I have programmed until know (it's functional):
- bootlader
- basic CPU initialization (IDT, GDT, protected mode...)
- graphic controller (simple VBE to have on screen messages)

Your wiki and Intel/AMD's developer manuals sure helped a lot for the early steps, but know I'm facing a problem which is twisting my mind a bit too much:
To start simple - "beginner mode" -, I would like to be sure my RTOS works in mono-core processing. Only when I'll fully understand everything about that would I go multi-threading and all.
For the sake of starting easy again, I'm using a BIOS to focus on the OS part at the moment. Here's where the problem starts: although BIOSes are not necessarily supposed to do so, I'm not entirely sure my BIOS does not startup all CPU AP cores.

However, neither in Intel Software Developer manual, nor in ACPI specifications, nor on OS Dev nor elsewhere did I find the way to "know" if CPU cores where enabled. Did I miss a flag or something ?

Nevertheless, I decided to proceed to shutdown every AP core according to Intel MP specifications: send the APs an INIT IPI with HALT instruction at warm reset vector.
But these specifications are quite old, and I remember reading on this forum this method was not done anymore. Is this still valid ?


Thanks a lot for your expertise !
(If ever this info was written somewhere, I guess I'll just go reprogram my brain)

At boot-time, you are guaranteed to be working in mono-core mode. You must explicitly turn on the other cores if you want to use them.

Ow, OK. Thanks then !

I guess it's true when they say the biggest problems are the ones you create yourself.


...just out of curiosity: how would you check the state of a core (i.e. enabled/disabled ?)

Hi,

mariuszp wrote:At boot-time, you are guaranteed to be working in mono-core mode. You must explicitly turn on the other cores if you want to use them.

That depends a little on how much you care about security.

I've always thought it'd be fun to have a "boot manager"/MBR that hooks "int 0x15" and steals some memory, then starts an AP CPU and modifies ACPI tables to remove any mention of that AP CPU, then chain-loads/boots the unsuspecting OS's boot loader. This way my code would have full access to the victim OS's physical memory and could do whatever it pleased with it; and most operating systems wouldn't detect my presence.

Tweety wrote: -> Is there a way to know if an AP core of a multi-core CPU has been enabled ?

No, not unless you started it or you know something specific about the code that did start it.

Tweety wrote: -> Is there a way to switch from multi-core to mono-core processing at OS startup? (shutdown/disable AP cores ?)

Yes, but I'm not too sure how "failure proof" it is. The basic idea is to send an INIT IPI to the CPU (causing it to return to a "wait for SIPI state"); but (due to various CPU errata, etc) I'd want take various precautions, like disabling paging, returning to real mode, disabling and flushing caches, etc.

Note that it may be possible for the BSP to send an INIT IPI to "all excluding self" and cause any/all AP CPUs that were started to be stopped (to enter that "wait for SIPI state"), without knowing if any AP CPUs were started or not. However I'm not sure how many different corner-cases might exist that could make this idea unsafe or unwise.

mariuszp wrote:For the sake of starting easy again, I'm using a BIOS to focus on the OS part at the moment.

The only reason to allow yourself to be constrained by the restrictions of BIOS is to make sure that you are unable to learn anything that you need to learn to achieve anything useful. It's like an athlete who decides the best way to start training to win a marathon is have the lower half of their body encased in solid concrete.

mariuszp wrote:Nevertheless, I decided to proceed to shutdown every AP core according to Intel MP specifications: send the APs an INIT IPI with HALT instruction at warm reset vector.
But these specifications are quite old, and I remember reading on this forum this method was not done anymore. Is this still valid ?

Once upon a time (a long time ago) the local APIC was an separate chip (not built into the CPU at all) and had some differences. One of those differences is AP initialisation; where you'd send "INIT IPI (assert)" then "INIT IPI (deassert)" to reset it, and it'd begin executing at firmware's rest vector.

None of that applies to Pentium or later CPUs (which have the local APIC built into the same chip as the CPU). For these you send an "INIT IPI" and then one or 2 "Startup IPIs"; where the CPU begins executing at the address determined by the "Startup IPI" (and doesn't use the firmware's reset vector).


Cheers,

Brendan

[EDIT: my main questions have been answered, this is more about curiosity]

Thanks a lot for the detailed explanations !

Brendan wrote:That depends a little on how much you care about security.

I've always thought it'd be fun to have a "boot manager"/MBR that hooks "int 0x15" and steals some memory, then starts an AP CPU and modifies ACPI tables to remove any mention of that AP CPU, then chain-loads/boots the unsuspecting OS's boot loader. This way my code would have full access to the victim OS's physical memory and could do whatever it pleased with it; and most operating systems wouldn't detect my presence.

That's pretty interesting, but wouldn't you need physical presence to inject that code anyway ?

Brendan wrote:

Tweety wrote:For the sake of starting easy again, I'm using a BIOS to focus on the OS part at the moment.

The only reason to allow yourself to be constrained by the restrictions of BIOS is to make sure that you are unable to learn anything that you need to learn to achieve anything useful. It's like an athlete who decides the best way to start training to win a marathon is have the lower half of their body encased in solid concrete.

You're probably right, I should do it, but I am a naughty Tweety.
I have more or less always coded for really low level soft, so I was expecting the BIOS to help me step up a bit. I guess you broke my dreams, going to work on my own "BIOS" then !

Hi,

Tweety wrote:

Brendan wrote:That depends a little on how much you care about security.

I've always thought it'd be fun to have a "boot manager"/MBR that hooks "int 0x15" and steals some memory, then starts an AP CPU and modifies ACPI tables to remove any mention of that AP CPU, then chain-loads/boots the unsuspecting OS's boot loader. This way my code would have full access to the victim OS's physical memory and could do whatever it pleased with it; and most operating systems wouldn't detect my presence.

That's pretty interesting, but wouldn't you need physical presence to inject that code anyway ?

You'd need some way to replace "boot manager"/MBR. One way to do this (without requiring physical access) would be to put a specially modified "GRUB installer disk image" online and trick people into using it.

Of course there might be practical uses too; like using the same technique as the basis for some kind of a debugging tool (to help people debug their hobby OS on real hardware).

Tweety wrote:

Brendan wrote:The only reason to allow yourself to be constrained by the restrictions of BIOS is to make sure that you are unable to learn anything that you need to learn to achieve anything useful. It's like an athlete who decides the best way to start training to win a marathon is have the lower half of their body encased in solid concrete.

You're probably right, I should do it, but I am a naughty Tweety.
I have more or less always coded for really low level soft, so I was expecting the BIOS to help me step up a bit. I guess you broke my dreams, going to work on my own "BIOS" then !

Using the BIOS in a boot loader is fine (and is often better than not using it). Using it after boot is where it becomes a problem.


Cheers,

Brendan

A number of rootkits, especially botnet rootkits, do exactly this, if I am not mistaken. And some of them can self-install in a drive-by download. Fun, right?

Schol-R-LEA wrote:A number of rootkits, especially botnet rootkits, do exactly this, if I am not mistaken. And some of them can self-install in a drive-by download. Fun, right?

I've seen those before but I've never seen one that pretends the affected core is actually there, and if strange things were happening on a computer and Task Manager said "you have three cores, isn't that wonderful?" on an i5 or i7 I'd be incredibly suspicious.

The really hard part is pretending there's a working core there and intercepting scheduling attempts on it.