could someone please clarify this for me? (memory paging)

[xDDunce]

hi,

I'm new to osdev.org and to osdev'ing! I finally got fed up of programming windows applications and decided to start making my own OS. so far I've completed the famous Bran's kernel development tutorial and have went on to bigger and better things

I'm currently writing a memory manager for my OS and and have ran into a little trouble.(i followed 2 tutorials found here and here to set up paging and all went well! as far as i know....)

i read the tutorials and i think i understand........ but i have a few questions to make sure.

1) having setup paging in the above tutorials, is the kernel already paged?(as the tutorials page 4mb already and this is much bigger than my kernel) if not, does it need to be paged? and how do i do it?(I'm not asking for code, because i like a challenge but cant figure this out without a hint so just a little guidance will help a lot).

2) how can i go about allocating memory? at the moment i can find an empty page and mark it as used, but where do i go from here? having read several tutorials, they say to return the address of the page. how do i do this and what should i do from there? the whole virtual to physical memory mapping is making my head go into overdrive!

thanks in advance,

James.

[Stevo14]

I'm currently writing a memory manager for my OS and and have ran into a little trouble.(i followed 2 tutorials found here and here to set up paging and all went well! as far as i know....)

i read the tutorials and i think i understand........ but i have a few questions to make sure.

Those two tutorials are basically the same thing but regardless, It is good that you understand them. Paging is a more advanced concept. But before I go on I would like to point out that you should first write a physical memory manager before you write a virtual memory manager (paging). A physical memory manager just keeps track of the areas of physical memory that are either free or used. It doesn't actually access physical memory, it just keeps track of what it looks like. Start with Memory_management and then see Page_Frame_Allocation. Remember, a "page frame" is simply a chunk of memory of a predefined size (4kb is the default on x86 systems).

1) having setup paging in the above tutorials, is the kernel already paged?(as the tutorials page 4mb already and this is much bigger than my kernel) if not, does it need to be paged? and how do i do it?(I'm not asking for code, because i like a challenge but cant figure this out without a hint so just a little guidance will help a lot).

Yes the kernel is paged. It appears in virtual memory at the same place that it actually is located in physical memory (grammar check?). This is called "Identity Mapping".

2) how can i go about allocating memory? at the moment i can find an empty page and mark it as used, but where do i go from here? having read several tutorials, they say to return the address of the page. how do i do this and what should i do from there? the whole virtual to physical memory mapping is making my head go into overdrive!

Your physical memory manager should be able to return a number to you that represents the starting address of a free piece of physical memory (henceforth referred to as a "page frame"). You then put that number into an index in the page table depending on which virtual page frame you would like to be translated to the physical page frame that your physical memory manager returned.

For example:
Let's say that your kernel needs to have a place to store dynamic data (it probably will sooner or later). For simplicity, you have set aside the memory from 0x10000000 to 0x1000FFFF (or whatever you choose) as the area where you will store this dynamic data. First you ask your physical memory manager for a number representing a free piece of physical memory:

Code: Select all

phys_address = phys_alloc(size_of_memory_needed);

You then tell you virtual memory manager that all memory access to the page frame at 0x10000000 should be translated to the physical page frame that we just found.

Code: Select all

virt_address = 0x10000000;
page_map(virt_address, phys_address);

Now it's your job to fill in those functions.

[xDDunce]

thanks a lot. this should keep me busy for the next 2 weeks before i go back to college

Those two tutorials are basically the same thing but regardless, It is good that you understand them

yes i know they are the same but they cover the same idea from different ideas, and having understood both, i implemented a version which i had the most understanding of and apparently it works so I'm proud

are there any more tutorial that could help?(i have a huge thirst for knowledge )

thanks for the help!

James.

[AJ]

Hi,

Have a look at http://www.jamesmolloy.co.uk - he has written a very good series of tutorials including memory management tutorials.

Cheers,
Adam

[xDDunce]

thanks AJ i followed the JamesMolloy tutorials and they were quite easy to understand, although not as easy to incorporate in my kernel.


i finnally managed to do it, and i ran into a triple fault. a bit of simple debugging led me to find the source:

Code: Select all

void switch_page_directory(page_directory_t *dir)
{
    current_directory = dir;
    asm volatile("mov %0, %%cr3":: "r"(&dir->tablesPhysical));
    u32int cr0;
    asm volatile("mov %%cr0, %0": "=r"(cr0));
    cr0 |= 0x80000000; // Enable paging!
    asm volatile("mov %0, %%cr0":: "r"(cr0));
}

ok. so i know that the error is in that piece of code (it is the last function called before returning to main()).

i discovered that the error is in the last line (i paused it by running bran's timer_wait()) so i was wondering if anyone knew how to fix this? i have absolutely no idea (im nowhere near as good at ASM as i am at C++) what is wrong with it.

also when i pause it, the pause runs forever but i know that interrupts are still being fired as i have a clock running (only a kernel uptime).

any ideas?

James.

[thepowersgang]

This error could be caused by the next instruction after setting cr0 not being mapped in memory. This causes a page fault. The triple is becuase the Error handling routines (isrs) are also not mapped. Check if the page dir you are passing is mapped correctly.

[xDDunce]

ok i found the problem....... Micosoft.

i have been testing my OS in MS Virtual PC, and last night (bout 2am) i decided to see if that was my problem so i tryed it out in bochs.

bochs ran it perfectly! then i decided to run it on real hardware and it worked perfect too! so i will not be using MS virtual PC anymore.

thanks for all the help tho! And now for a new problem

i have followed JamesMolloy's "The Heap" tutorial and ran into some trouble initialising paging (again - but i worked out its in the new code).

Code: Select all

heap_t *create_heap(u32int start, u32int end_addr, u32int max, u8int supervisor, u8int readonly)
{
    heap_t *heap = (heap_t*)kmalloc(sizeof(heap_t));
    // All our assumptions are made on startAddress and endAddress being page-aligned.
    ASSERT(start%0x1000 == 0);
    ASSERT(end_addr%0x1000 == 0);
    // Initialise the index.
    puts("section1");
    heap->index = place_ordered_array( (void*)start, HEAP_INDEX_SIZE, &header_t_less_than);
    puts("section2!");
    // Shift the start address forward to resemble where we can start putting data.
    start += sizeof(type_t)*HEAP_INDEX_SIZE;

    // Make sure the start address is page-aligned.
    if ((start & 0xFFFFF000) != 0)
    {..........
      

the above code only prints section1 then throws a page fault. it has nothing to do with place_ordered_array() as i have a completion message a the end to tell me when it has done it so it has to be in the assignment to heap->index. any ideas?

James.

[xDDunce]

ok. i got my code to complete with out a page fault (finally!) but im not sure if it is correct.

if i change the first line of the function into 2:

Code: Select all

heap_t *heap;
kmalloc(sizeof(heap));

would this work the same? i got it to go through the function without another page fault and my interrupts are running(PIT and keyboard) but is it right?

James.

[neon]

ok i found the problem....... Micosoft.
i have been testing my OS in MS Virtual PC, and last night (bout 2am) i decided to see if that was my problem so i tryed it out in bochs.

bochs ran it perfectly! then i decided to run it on real hardware and it worked perfect too! so i will not be using MS virtual PC anymore.

If it fails to work on Virtual PC then it is a compatibility issue that should be resolved. Just because it works on some hardware (and Bochs) does not mean it will work on others. You should test your system on as many different configurations as possible and insure it works.

[bewing]

would this work the same?

Uh ... no, it shouldn't work the same at all. sizeof(heap) is going to be either 4 or 8, because heap is just a pointer. sizeof(heap_t) will certainly be something completely different.

And I'm surprised it doesn't crash after that, because heap is being used uninitialized -- since you never set it to the return value of kmalloc?

[xDDunce]

Ah, yes of course. sorry that was one heck of a late night...

it does run properly, although i do see what is wrong with the code. thanks for the clarification though!

is there any other way of making this code work? or will i have to rewrite the entire heap section?

James.

[JamesM]

Hi,

As I wrote the code, and others have tested it too, I know that the code itself /does/ actually work.

So the answer is to debug effectively. Get your page fault handler to print out the EIP that faulted and why it faulted - read/write? and the address it faulted at.

Then, compare it with a disassembly of your kernel to find where it occurred. If in doubt, you can post the disassembly along with the page fault output.

You should also check the return value from kmalloc. Does it look sane? it should be between 0x100000 and 0x200000.

Cheers,

James

[xDDunce]

i havent touched it for a few days now so i will get back to it real soon although i need a new development host as my laptop just BSoD-ed and now wont boot any internal devices (cracked motherboard suspicion) but will boot my external floppy drive so atleast its useful for one thing eh?

anyway.... i vaguely remember the code erroring because kmalloc returns around the 0xC000000(0) ish area (not sure how many zeroes but i think it was 7) so this would of course raise a page fault.

i'm getting a macbook soon, a gift from my parents for passing my GCSEs and i'll need to get an old pc to get all my sources off of the hard disk(i dont wanna risk anything with a new PC even though i know how easy it is - i'm very accident prone ) then i will post back with results.

thanks for all the help!

James.

[Adek336]

My Dell laptop won't boot from anything other than the NIC card if the hard drive is broken or removed (probably buggy BIOS); as I understand it, both a NIC and an external floppy supply the BIOS with booting code in a ROM so it seems quite simillar so I suggest you check you hard drive.

[xDDunce]

thanks for the info. my friend has donated his old linux box for me to check the hard drive/recover files so i will be able to find out soon.

James.