Paging, and interpretation of the Legacy memory map

[Schol-R-LEA]

Following on from an earlier discussion here, I am working on replacing the stub paging from my bootloader with kernel-defined paging, specifically with regards to the ACPI firmware tables.

I was hoping to determine the location of said firmware and related MMIO from the memory map I retrieved in the boot loader using INT 15h, EAX=0xE820. Unfortunately, the map has limited (for lack of a better term) resolution regarding what is where - it shows 'free' and 'reserved' memory regions, but doesn't indicate what each reserved section contains.

I know empirically from my code instrumentation (e.g., debugging print outs, QEMU logs) where the specific ACPI RSDT I need is, but that doesn't really solve the problem - how to use the memory map to locate specific firmware and hardware mappings. I can only assume I am missing, or misunderstanding, something.

On a related note, I am finding myself in a position where I am going to need to start managing free memory, but I am uncertain if it makes sense to create a kernel memory manager before I have demand paging working. The two issues seem to be independent of each other, at least in a naive implementation, but I can't help feel uncertain if one has to come before the other.

On a final note, I am also looking to remap the kernel stack to a location set in my linker script. I'm not certain just how to ensure that this will work without smashing the existing stack.

Please bear with me, I am sorry if my anxieties are getting the better of me on these matters.

[Gigasoft]

The memory map is only for reporting usable memory.

As described in the ACPI specification, to find the ACPI RSDT, look for "RSD PTR " in the EBDA, and if it isn't there, search 0e0000h-0fffffh, every 16 bytes. This contains a pointer to the RSDT at offset 12.

ACPI gives you the address of fixed function registers that it defines, whereas other hardware is detected through PCI and recognized based on vendor/device ID or class code, with addresses defined in PCI BARs. ACPI can also declare devices with a PNP ID and a resource descriptor.

[sounds]

To expand a little more on this -

The memory map is only for reporting usable memory.

As described in the ACPI specification, to find the ACPI RSDT

It is unfortunately true that the E820 memory map can't be used to figure out which pages to map.

A typical approach then is to map pages as needed, on demand.

Load the "RSD PTR " (aka RSDP) to find the RSDT. Since you know about what size it is, map those bytes into your virtual address space and then parse the RSDT. That may take one page, or two if the RSDT crosses a page boundary. (Highly unlikely, maybe you prefer to just refuse to boot if the RSDT crosses a boundary.)

From the RSDT you will find more tables. For each one, figure out how big it is, then go and do a page mapping on that physical address, if needed. Hopefully many of the tables fall into the same small number of pages.

Keep in mind this is what ACPICA does, and you may find it easier to just link to that. Or, more specifically, ACPICA expects the underlying OS to provide it with hooks to map pages, and then ACPICA makes sure to map the virtual memory before each table is loaded.

[Octocontrabass]

I can only assume I am missing, or misunderstanding, something.

The memory map isn't supposed to tell you what's in the reserved regions, just that they exist and they're not usable RAM. That way, you won't try to assign some other hardware to that address (using PCI BARs or whatever), and you won't try using that address as RAM.

If you need to access those reserved memory regions for some reason, you'll find out some other way. For example, the ACPI specification says to search a couple of those reserved regions for the RSDP (unless you're using UEFI), and the RSDP will tell you that the RSDT or XSDT is also located in a reserved region, and so on. You don't always need to map the entire reserved region, just enough to access what you're looking for.

On a related note, I am finding myself in a position where I am going to need to start managing free memory, but I am uncertain if it makes sense to create a kernel memory manager before I have demand paging working. The two issues seem to be independent of each other, at least in a naive implementation, but I can't help feel uncertain if one has to come before the other.

What would you do with demand paging without memory allocation? Your kernel should always know beforehand what it needs to access...

On a final note, I am also looking to remap the kernel stack to a location set in my linker script. I'm not certain just how to ensure that this will work without smashing the existing stack.

Easy: do it while there's nothing in the stack. For example, right before your entry code calls kernel_main().

[Schol-R-LEA]

To expand a little more on this -

The memory map is only for reporting usable memory.

As described in the ACPI specification, to find the ACPI RSDT

It is unfortunately true that the E820 memory map can't be used to figure out which pages to map.

A typical approach then is to map pages as needed, on demand.

Load the "RSD PTR " (aka RSDP) to find the RSDT. Since you know about what size it is, map those bytes into your virtual address space and then parse the RSDT. That may take one page, or two if the RSDT crosses a page boundary. (Highly unlikely, maybe you prefer to just refuse to boot if the RSDT crosses a boundary.)

From the RSDT you will find more tables. For each one, figure out how big it is, then go and do a page mapping on that physical address, if needed. Hopefully many of the tables fall into the same small number of pages.

Ah, OK then. I take it that the RSDT (or XSDT in ACPI >1.0) is more or less the key to determining what other pages need to be mapped, or at the very least, where several key ones can be found?

Keep in mind this is what ACPICA does, and you may find it easier to just link to that. Or, more specifically, ACPICA expects the underlying OS to provide it with hooks to map pages, and then ACPICA makes sure to map the virtual memory before each table is loaded.

As tempting as it is, I think I need to go through the process manually to so I can understand what is actually going on.

[Schol-R-LEA]

On a related note, I am finding myself in a position where I am going to need to start managing free memory, but I am uncertain if it makes sense to create a kernel memory manager before I have demand paging working. The two issues seem to be independent of each other, at least in a naive implementation, but I can't help feel uncertain if one has to come before the other.

What would you do with demand paging without memory allocation? Your kernel should always know beforehand what it needs to access...

On a final note, I am also looking to remap the kernel stack to a location set in my linker script. I'm not certain just how to ensure that this will work without smashing the existing stack.

Easy: do it while there's nothing in the stack. For example, right before your entry code calls kernel_main().

Ah, that makes sense. I assume I would need to map the intended location in the boot loader paging stub, however.

[Octocontrabass]

I assume I would need to map the intended location in the boot loader paging stub, however.

Your bootloader maps memory according to the program headers, right? Just make sure there's a program header that covers the stack area.

[Schol-R-LEA]

On the matter of the stack, I have changed the link script thusly:

Code: Select all

    /* set up the kernel stack well above the system tables */
    . = 0xC1000000;

    .stack :
	{
		kernel_stack_base = .;
		kernel_stack_top = . + (4M - 4);
	}

and in kstart.asm:

Code: Select all

   extern kernel_main
   extern kernel_stack_top
   global kstart

[SECTION .text]
kstart:
        mov eax, kernel_stack_top
        mov esp, eax

        call kernel_main

   .halted_loop:
        hlt
        jmp short .halted_loop

There were no regressions from this change, so I assume that this is more or less what Octocontrabass had in mind, but I felt the need to confirm this.

Again, please be patient regarding my anxiety.

[Octocontrabass]

Code: Select all

		kernel_stack_top = . + (4M - 4);

Code: Select all

        mov eax, kernel_stack_top
        mov esp, eax

        call kernel_main

The System V psABI requires the stack to be 16-byte aligned before making a function call. Subtracting 4 here misaligns your stack; subtract 0 instead. (Or tell your compiler that kernel_main() is called with a misaligned stack.)

Also, ESP is a general-purpose register, you can MOV directly into it. Also, 4 megabytes is extremely large for a kernel stack - 4 kilobytes is usually enough.

[Schol-R-LEA]

Ah, thank you. I'll make the changes you are recommending now.