Getting in paging

[Thpertic]

I'm following Tim Robinson's tutorial on osdever. I understood it, but now I'm not sure how to implement it. I already have paging enabled due to the higher half kernel I set up, so should I simply create a Page directory and put it where I told cr3 it was, or just do another table and tell cr3 where it stands? The higher half tutorial implement 4MB paging, but now I think that 4KB is better. Should I reset cr4 after the higher half is set up or I just set it before?
To use a stack (as the tutorial states it is better) should I set like esp to the end of the kernel and push there? Do I push just the first address of the page and then jump to the other page address to push?

Do you have any other tutorial or some easy-to-read code of this topic?
Thanks

[Brendan]

Hi,

I'm following Tim Robinson's tutorial on osdever. I understood it, but now I'm not sure how to implement it. I already have paging enabled due to the higher half kernel I set up, so should I simply create a Page directory and put it where I told cr3 it was, or just do another table and tell cr3 where it stands? The higher half tutorial implement 4MB paging, but now I think that 4KB is better. Should I reset cr4 after the higher half is set up or I just set it before?

For which purpose? For most things, you allocate/obtain a page directory once, then make modify the existing page directory whenever you want to add/remove page tables and add/remove pages.

To use a stack (as the tutorial states it is better) should I set like esp to the end of the kernel and push there? Do I push just the first address of the page and then jump to the other page address to push?

There's the general concept of "a stack", which is any kind of data structure that implements the features of a "last in first out" buffer where it's to add a new thing to one end ("push" something on the stack) and easy to retrieve one thing from the same end ("pop" something off the stack). There are many concrete implementations of a stack. One example of a stack is the CPU's stack, which uses ESP to keep track of where new things will be added/pushed and where things will be retrieved/popped. Another example of a stack is a singly linked list to manage a pool of free pages, where the first few bytes of each page contains a link/pointer/address of the previous page in the linked list. Even though these things use completely different implementations and are used for completely different purposes, they're both a kind of stack (or both a kind of "last in first out" buffer).

For the CPU's stack there's no sane alternatives - you mostly have to use it, and have to have ESP pointing at an area of "read/write" RAM to be able to use it. It doesn't make sense to say "the CPU's stack is better" (because there's nothing to compare it to - better than what?), which is why I think you're confusing "CPU stack" with "a free page stack".

For a singly linked list to manage a pool of free pages (a free page stack) there's multiple other alternatives to manage free pages, and it does make sense to say "it's better (than some or all alternatives, in some way/s but not others)"; but this would have nothing to do with the CPU's stack and wouldn't have anything to do with ESP either.

Do you have any other tutorial or some easy-to-read code of this topic?

I'd recommend starting with this guide to get a broad overview of what the goals are; then maybe this page about paging in case there's something you've overlooked or not sure of, and maybe this page about memory management to get an idea of some of the different ways that physical memory management and virtual memory management could be done.


Cheers,

Brendan

[Thpertic]

Ok, so I use the BootPageDirectory set in the bootloader as page directory. Can you help me in how to add page tables in the page directory? And I have another question... I should add to the stack (using the BIOS memory map) like 0x0000 (if it's free) and the next frame at like 0x0400 (if pages are 4kb)?

[isaacwoods]

... and the next frame at like 0x0400 (if pages are 4kb)?

How did you get this number? 4096 == 0x1000

[Thpertic]

How did you get this number? 4096 == 0x1000

Oh my bad... that's 1024, sorry

[Brendan]

Hi,

Can you help me in how to add page tables in the page directory?

If the physical page containing the page directory is mapped into the virtual address space somewhere; "adding page tables" means finding/obtaining/allocating a physical page somehow, setting some flags in the physical address, calculating the virtual address of the page directory entry, then writing the "page table physical address + flags" value to the "virtual addresss of page directory entry".

And I have another question... I should add to the stack (using the BIOS memory map) like 0x0000 (if it's free) and the next frame at like 0x0400 (if pages are 4kb)?

Most people have nested loops, like:

• For each entry in the memory map:
  • If entry type is "usable RAM":
    • Align start and end of the area described by the entry to a page boundary (round "start" up, round "end" down).
    • For each page from "start" to "end":
      • Add page to free page stack (probably by "freeing" the physical page)

However, you do also need to do something to avoid adding pages of usable RAM that are already being used to the stack of free physical pages. There's 3 ways to do this - modify the memory map to mark those areas as "usable but not free", do a check inside the "for each page from start to end" loop (to decide if the page should/shouldn't be added the page to the free page stack) , or subtract used areas from the larger area (potentially causing a larger are to be split into two smaller ranges because RAM is used in the middle) before you do the "for each page from start to end".

How you do this is up to you. I use the first option (I convert the memory map into my own completely different format that includes a lot more information) but it's not as easy. The second option (a check for every page) is inefficient but also a lot easier to implement (especially when the memory you're already using is "scattered" in many areas). The third option is a efficient but a little tricky (there's various corner cases that you need to handle correctly, potentially including "large area fragmented into many small pieces").


Cheers,

Brendan

[Thpertic]

I found out a code about the int 0x15 on a page you redirected me and I translated it to AT&T, the problem is that qemu reboots when I call the function in the first

Code: Select all

movb $1, %es:20(%di)

Part of Boot.S:

Code: Select all

# This function will get the physical memory map from the BIOS
# usign InT 0x15, EAX = 0xe820
.global getPhysicalMemory
.extern mmap_ent

getPhysicalMemory:
	xor %ebx, %ebx		        # ebx must be 0 to start
	xor %bp, %bp	            # keep an entry count in bp
	mov $0x0534D4150, %edx	    # Place "SMAP" into edx
	mov $0xe820, %eax
                    # CPU FAULTS /*/**/*/***/
	movb $1, %es:20(%di)       	# force a valid ACPI 3.X entry -- mov [es:di + 20], 1
	hlt
    mov $24, %ecx		        # ask for 24 bytes
	
    int $0x15
    
	jc .failed	                # carry set on first call means "unsupported function"
	mov $0x0534D4150, %edx	    # Some BIOSes apparently trash this register?
	cmp %edx, %eax		        # on success, eax must have been reset to "SMAP"
	jne .failed
	test %ebx, %ebx		        # ebx = 0 implies list is only 1 entry long (worthless)
	je .failed
	jmp .jmpin
.e820lp:
    mov $0xe820, %eax           # eax, ecx get trashed on every int 0x15 call
	movb $1, %es:20(%di)       	# force a valid ACPI 3.X entry
	mov $24, %ecx		        # ask for 24 bytes
	int $0x15
	jc .e820f	    	        # carry set means "end of list already reached"
	mov $0x0534D4150, %edx	    # repair potentially trashed register
.jmpin:
	jcxz .skipent		        # skip any 0 length entries
	cmp $20, %cl	            # got a 24 byte ACPI 3.X response?
	jbe .notext
	testw $1, %es:20(%di)      	# if so: is the "ignore this data" bit clear?
	je .skipent
.notext:
	mov %es:8(%di), %ecx 	# get lower uint32_t of memory region length
	or %es:12(%di), %ecx 	# "or" it with upper uint32_t to test for zero
	jz .skipent		            # if length uint64_t is 0, skip entry
	inc %bp			            # got a good entry: ++count, move to next storage spot
	add $24, %di
.skipent:
	test %ebx, %ebx		        # if ebx resets to 0, list is complete
	jne .e820lp
.e820f:
	mov %bp, (mmap_ent)	        # store the entry count
	clc			                # there is "jc" on end of list to this point, so the carry must be cleared
	ret
.failed:
	stc			                # "function unsupported" error exit
	ret

Can you help me?

[MichaelPetch]

As I recall you are in protected mode so you won't be able to use the BIOS interrupts so that will crash. I recall you were the fellow who was doing a higher half kernel using Multiboot. You can get Multiboot loader (ie GRUB etc) to give you a memory map that can be iterated over.

[Thpertic]

Yes, I am. Can you suggest me some good explanation of how the map is structured? I can't already understand it well. Also, I'd want to ask if 4MB pages (as I implemented while setting the higher half kernel) are right or I just have to switch to 4KB to get it better? Is it mandatory to use identity mapping? Thanks