Can't figure out paging

[Gigaboy]

I'm trying to add paging to my OS and I've tried several methods online, but to no avail. I already have a simple paging function (with free memory address hardcoded in) in mem.c but it's no where near as complex as the paging methods I've found on the internet. I was following this tutorial OS From Scratch but I finished his book and he doesn't have paging so next I started following JamesM's tutorial and I got stuck at paging. If somebody could point me to a paging tutorial with my own bootloader (instead of GRUB) I would like that. Or my files are at this dropbox link if you would please take a look at them.

Paging Method from libc/mem.c

Code: Select all

uint32_t free_mem_addr = 0x10000;

uint32_t kmalloc(uint32_t size, int align, uint32_t *phys_addr) {
	if (align ==1 && (free_mem_addr & 0xFFFFF000)) {
		free_mem_addr &= 0xFFFFF000;
		free_mem_addr += 0x1000;
	}

	if (phys_addr) *phys_addr = free_mem_addr;

	uint32_t ret = free_mem_addr;
	free_mem_addr += size;
	return ret;
}

[Octacone]

The function from above really doesn't have anything to do with paging at all.
I would advise you against following any tutorials, because that is one of the most crucial parts of an operating system.
Intel manuals are a good way to start: https://software.intel.com/sites/defaul ... -3abcd.pdf, take a look at Chapter 4 called Paging, it has everything you will need. It might take a few (x10) reads.
Paging is something very specific and requires a good understanding of the mechanism itself, before you can code anything.
You primary goals should be:
1.Figure out how it works (in details) in your head.
2.Figure out what a page directory, page directory entry, page table and page itself is. I would suggest you using bit-fields, leave clunky arrays of integers and bit-shifts alone.
3.Figure out how to translate an address aka how to map it. Physical -> Virtual
4.Enable paging by setting PG bit in CR0.
5.Tell us how badly does it triple fault, remember: paging structures don't like unaligned addresses. It has to fail at least once, it is perfectly normal to experience some issues at the very beginning.

This was just a really really short overview. You still have a lot of decision to make: higher half, PAE or not, PSE or not (you are not ready for this yet, not a good idea)...
Feel free to ask anything you want. Currently I don't have any more time but if you want I can explain you in detail how I did it and what paths did I take.

[Gigaboy]

Feel free to ask anything you want. Currently I don't have any more time but if you want I can explain you in detail how I did it and what paths did I take.

Do I need paging in my OS? I've seen some OSes without paging in it at all.

Thanks for the Intel link, it's very helpful.

[BrightLight]

Do I need paging in my OS? I've seen some OSes without paging in it at all.

Definitely.

Paging is the hardware's mechanism of memory protection, virtual address space, and other modern things. Of course, it's possible to write an OS that doesn't use paging, and depends either on segmentation or a flat non-paged address space. For the initial, you're using obsolete technology. Most OSes today don't use segmentation, and even x86_64 removed support for segmentation at all (except for FS and GS registers, but that's irrelevant.) For the latter, each program must be loaded at a different address, which means programs must be aware that they are loaded at a non-standard location, or that each program must have their own linker script, which is plain ridiculous.

In any case, in order to properly implement multitasking, which is required in any modern OS, you need to implement paging. Octacone's overview on paging should get you started, and the Intel manuals will help a lot. Feel free to ask here when you need help.

[Gigaboy]

In JamesM's tutorial he uses the variable "placement_address". It's defined in his kheap.c file

Code: Select all

// end is defined in the linker script.
extern u32int end;
u32int placement_address = (u32int)&end;

and "end" is defined in the linker script.

Code: Select all

end = .; _end = .; __end = .;

He's using GRUB for his bootloader and I'm using my own bootloader, my question is how to I get the value of "end" in my bootloader?

EDIT: More specifically, how do I set "placement_address" to the end of my kernel.

[glauxosdever]

Hi,

In JamesM's tutorial

I suggest you don't use JamesM's tutorial. It's full of bugs, hasn't been updated in 9 years and probably hasn't even been peer reviewed.

As for paging, yes, you probably need it if you want to make a modern OS. I however don't know of a good tutorial about paging - you might as well try Octacone's advice for now.


Regards,
glauxosdever

[Gigaboy]

I hate low-level coding and everything in the Intel manual is Assembly but I'll try to use that.

[iansjack]

Though the Intel manuals describe concepts in terms of the underlying hardware, it doesn't mean that you have to use assembler to realize those concepts. Paging requires no assembler instructions, apart from those to enable it in the first place and the occasional instruction to flush the TLBs and to load/save register cr3.

[lkurusa]

If you hate low-level coding, then why are you trying to do OS development? Just out of curiosity.

[Octacone]

There is almost no low-level assembly involved when it comes to paging. You won't need more than a few lines of code. - Very simple code
You definitely need paging, it is widely used these days, provides some mandatory features, also Long Mode has paging enabled by default.
You certainly won't be able to learn/understand it over night, it takes some time and deep concentration. The main problem I see is, you don't even know why you need paging.
First figure out what you can't do without it and then try to understand how to implement it.

[Gigaboy]

If you hate low-level coding, then why are you trying to do OS development? Just out of curiosity.

I enjoy coding and I thought it would be fun to make my own OS. (It is fun BTW)

There is almost no low-level assembly involved when it comes to paging. You won't need more than a few lines of code. - Very simple code
You definitely need paging, it is widely used these days, provides some mandatory features, also Long Mode has paging enabled by default.
You certainly won't be able to learn/understand it over night, it takes some time and deep concentration. The main problem I see is, you don't even know why you need paging.
First figure out what you can't do without it and then try to understand how to implement it.

I'll try to figure it out on my own but over the week I might post some questions here.

[Gigaboy]

After reading through 32-bit paging section in the Intel PDF file and looking at others source code I came up with the code in the files paging.c/h and kheap.c/h. All the C files compile with no errors/warnings and when I run the OS image in Qemu it doesn't work. The files I used for paging (paging.c/h, kheap.c/h, panic.c/h, and system.h) are all in the libc/ directory. Dropbox Files

[Octacone]

After reading through 32-bit paging section in the Intel PDF file and looking at others source code I came up with the code in the files paging.c/h and kheap.c/h. All the C files compile with no errors/warnings and when I run the OS image in Qemu it doesn't work. The files I used for paging (paging.c/h, kheap.c/h, panic.c/h, and system.h) are all in the libc/ directory. Dropbox Files

It is really a bad idea to use that code, as stated above the tutorial you're following is not the best. It can be used as a reference but nothing more.
Please take a look: http://wiki.osdev.org/James_Molloy's_Tu ... Known_Bugs, you really shouldn't be using that code.
You probably don't know it, but there is a bitmap based Physical Memory Manager integrated within that code, which is really really bad. PMM should be an entirely separate concept and it has nothing to with paging.
Why did you put your files inside a libc folder doe? That seems kind of (and is) really strange.
Please try to code it yourself. Don't copy and past code, you will learn very little by doing so.
It is much more likely that you will learn by trying to code something by yourself and encountering problems yourself.
Paging is not hard, it just required a certain level of logical understanding.

Other than that, what exactly is your problem. Doesn't work is not very descriptive.

[Gigaboy]

...what exactly is your problem. Doesn't work is not very descriptive.

Qemu keeps looping the code up to init_page()
5 sec video

It is really a bad idea to use that code, as stated above the tutorial you're following is not the best. It can be used as a reference but nothing more.
Please take a look: http://wiki.osdev.org/James_Molloy's_Tu ... Known_Bugs, you really shouldn't be using that code.
You probably don't know it, but there is a bitmap based Physical Memory Manager integrated within that code, which is really really bad. PMM should be an entirely separate concept and it has nothing to with paging.

I had no idea JamesM's tutorials had that many bugs (gonna stop following his tutorial).

Why did you put your files inside a libc folder doe? That seems kind of (and is) really strange.

Yes, it's strange. I was going to make a mm/ directory for paging, heap, malloc, etc. but I didn't feel like creating a new directory if the code wasn't going to work. My Makefile only compiles the code in libc/, cpu/, or kernel/ so I had to put the files in one of those folders or it wouldn't compile them.

[Octacone]

...what exactly is your problem. Doesn't work is not very descriptive.

Qemu keeps looping the code up to init_page()
5 sec video

That is definitely a Triple Fault, possibly followed by a page fault.
It is time for some debugging. Use Bochs and tell us the exact error message.
Possible causes (some of them):

• Page directory/entries are not page aligned.
• CR3 doesn't point to a page directory.
• Kernel is not mapped.
• Paging structures are not mapped.
• Trying to access invalid/(not mapped, not present) memory.

But we can only guess until you show us the real error.