Where should I put my paging structure?

[antoni]

Hi,

I'm working on my kernel that is running in single address space for now. For mapping pages I use this function: https://pastebin.com/LPwYrkcp

This function successfully maps (physical) memory between 0x200000 and 0x8000000 (first 512 pages are mapped in boot.asm when I'm entering long mode), but it fails to map anything above.
I.e. After running this function and touching cr3 I don't get #PF, but for example when I access this memory it always returns zero and qemu says:

(gdb) x/b 0x8000000
0x8000000: Cannot access memory at address 0x8000000

Since this function looks good to me I thought maybe my paging structure overlaps some other structures in memory. Root of my paging structure (PML4) is at 0x1000. So entry for 0x8000000 is at ps_root[34304] = 0x1000 + 34304 * 8 = 0x44000.

I wonder if there is something there.

[bzt]

I thought maybe my paging structure overlaps some other structures in memory.

You should implement kernel heap to avoid that. Because you'll always need an entire page to be allocated for the paging structures, you should use a page frame allocator. Alternatively you could use the normal kernel heap allocator, if you can pass alignment requirement to it.

Cheers,
bzt

[antoni]

I even have such a system. I have a bigger problem with completing the linked list of free physical memory.

GRUB's memory map doesn't give me any useful information: https://pastebin.com/WUq1rzZQ
I read it in such a way, but it is rather correct: https://pastebin.com/Y1wz47Ef

There are a lot of things in memory. Like ACPI structures, or the VGA buffer. ACPI structures should be listed in the GRUB's map, I don't know why they aren't.

[bzt]

I even have such a system. I have a bigger problem with completing the linked list of free physical memory.

I don't think anybody can help with that. Maybe first try to write a small program on your developer machine to figure out how to handle lists, and when it works, copy the code into your kernel. Using linked lists in C is basic and a required skill for OSDev.

GRUB's memory map doesn't give me any useful information: https://pastebin.com/WUq1rzZQ
I read it in such a way, but it is rather correct: https://pastebin.com/Y1wz47Ef

Of course, that's a memory map. You should get all the "available" records, put them into your allocator's free memory list on init, and forget about the other records (from the allocator's point of view).

There are a lot of things in memory. Like ACPI structures, or the VGA buffer. ACPI structures should be listed in the GRUB's map, I don't know why they aren't.

Because it's just an E820 map. Not all BIOS adds an ACPI record, you should look for RSDP structure in the EBDA. That structure will hold a pointer to the root of the ACPI tables (either RSDT or XSDT). You might want to consider BOOTBOOT, that returns a parsed, standardized and ordered memory map no matter the platform, and the ACPI pointer in bootboot.x86_64.acpi_ptr as well (no need to search for it on BIOS and works with UEFI too).

Cheers,
bzt

[antoni]

I don't think anybody can help with that. Maybe first try to write a small program on your developer machine to figure out how to handle lists, and when it works, copy the code into your kernel. Using linked lists in C is basic and a required skill for OSDev.

I meant that I already have a working system, but I don't have a memory map from which I can insert values there.

Because it's just an E820 map. Not all BIOS adds an ACPI record, you should look for RSDP structure in the EBDA. That structure will hold a pointer to the root of the ACPI tables (either RSDT or XSDT). You might want to consider BOOTBOOT, that returns a parsed, standardized and ordered memory map no matter the platform, and the ACPI pointer in bootboot.x86_64.acpi_ptr as well (no need to search for it on BIOS and works with UEFI too).

This BOOTBOOT seems very nice. Only, does it list EVERY structure in memory, like VGA memory, or BIOS memory, and also memory holes, for example, does it list the ISA Memory Hole? If one memory region is marked "AVAILABLE" in this map, does that mean there are no ACPI structures or anything else there and I can use it entirely for storing data (and it exists)?

[bzt]

I meant that I already have a working system, but I don't have a memory map from which I can insert values there.

Oh, I see. Then sorry, I misunderstood.

This BOOTBOOT seems very nice. Only, does it list EVERY structure in memory, like VGA memory, or BIOS memory, and also memory holes, for example, does it list the ISA Memory Hole?

Nope, it only guarantees that all the free memory areas are listed, and independently to the memory map, pointers to system structures also returned. (However on BIOS, it verifies and fixes the map if necessary.)

If one memory region is marked "AVAILABLE" in this map, does that mean there are no ACPI structures or anything else there and I can use it entirely for storing data (and it exists)?

Yes, absolutely. This shouldn't be a problem in the first place, but BOOTBOOT does extra checks to only return valid map on buggy BIOSes too. If the map marks a region as free and usable, you can use it for storing data for sure.

Cheers,
bzt