[SOLVED] Memory Map from GRUB is complete, Right?

[ashishkumar4]

Here is another stupid post and please bear it as you did my last posts :p sorry again for disturbing.
But this time,
I want to ask wether the Memory Map GRUB gives us is complete or not? Don't Jump to conclusion, please read the query completely :3

I have successfully got the memory map from grub which I used to make memory blocks (each of 4096 bytes size) which I then use in my virtual memory allocation system. Now what I do is I just tested wether the memory which GRUB Memory map said its available (type 1) is actually working or not. I just Identity mapped the whole memory (and didn't did anything else before it), enabled paging and then tested the virtual mem system. I Tested the system by simply writing and reading from the memory which a block contained. I could successfully Read ALL available memory in QEMU (with 1 GB mem ) without any trouble. But when I try the SAME thing with VMWare (with 3.5GB mem, 3.3 GB available from first 1 MB to 3.3rd GB straight as GRUB says), I get Page Fault at 1.1GB (sorry for not using hex, its 3219062784) and the system crashes. The Page Fault Handler I used displayed the address and reason for page fault. It said that the page fault was due to READ ONLY page but it didn't displayed the physical memory address of the fault. But because I was displaying every memory address which I checked, I got to know the memory location was 11016561968 (bytes) though the last mem location which should be accessible is 3219062784. I concluded the error was because the frame the page refered to at 11016561968 was not available (the memory wasn't there). If it is so OR if itsnt so, then too, why the Memory Map GRUB provides in VMWARE is not complete while everything is FINE in qemu??

Or if I am missing something, please do tell me

Reply politely, because I know I am an IDIOT :3 :p Thanks for helping

EDIT: VIRTUAL BOX SHOWING SAME PROBLEM

Memory Map from Virtual Box:
start: 0 size: 652288 type 1
start: 652288 size: 3072 type 2
start: 817392 size: 8192 type 2
start: 901120 size: 147456 type 2
start: 1048576 size: 3219062784 type 1
start: 3220111360 size: 61440 type 3
rest if needed, ask me.

Memory Map from GRUB IN VIRTUALBOX:
start: 0 size: 652288 type 1
start: 652288 size: 3072 type 2
start: 817392 size: 8192 type 2
start: 901120 size: 147456 type 2
start: 1048576 size: 3333357568 type 1
start: 3334406144 size: 65535 type 3
rest if needed, ask me.

Memory Map from GRUB IN QEMU:
start: 0 size: 652288 type 1
start: 652288 size: 3072 type 2
start: 817392 size: 8192 type 2
start: 901120 size: 147456 type 2
start: 1048576 size: 107255808 type 1
start: 1073606656 size: 135168 type 2
rest if needed, ask me.

Paging Code:

// The size of physical memory.
u32int mem_end_page=(0xFFFFFFFF-(4096*1024));
nframes = mem_end_page / 0x1000;
frames = (u32int*)kmalloc(INDEX_FROM_BIT(nframes));
memset(frames, 0, INDEX_FROM_BIT(nframes));

u32int phys;
kernel_directory = (page_directory_t*)kmalloc_a(sizeof(page_directory_t));
memset(kernel_directory, 0, sizeof(page_directory_t));
kernel_directory->physicalAddr = (u32int)kernel_directory->tablesPhysical;

for(uint32_t i=0;i<(1024*maxmem);i+=4096) //page virtually every available free memory. maxmem is Memory size in KB from GRUB
alloc_frame(get_page(i,1,kernel_directory),1,1);

int i = 0;
switch_page_directory(kernel_directory);
current_directory = clone_directory(kernel_directory);
switch_page_directory(current_directory);

[shmx]

I did not get anything. Question about E820 function? Show memory map allocation. I had no problems with VMware.

[Nable]

Could you print the memory map from both VMs and compare it manually?
You can also post it here, maybe someone would notice the differencies.
Your code for setting up page tables may also be useful to find the cause of your problem.

[ashishkumar4]

Every Information I thought is necessary, I have posted above, Edited my post.

[Combuster]

You still seem to make the fallacy that page faults are caused by something other than page tables. You say you mapped all of memory, yet you still get a pagefault. What does GRUB have to do with that? What does the memory map have to do with that? It doesn't sound like you have even tried to reduce the problem.

[shmx]

for(uint32_t i=0;i<(1024*maxmem);i+=4096) //page virtually every memory!!! maxmem is Memory size in KB from GRUB
alloc_frame(get_page(i,1,kernel_directory),1,1);

How to calculate maxmem? Show source code function alloc_frame, get_page.

[ashishkumar4]

Combuster sir, I am not telling that from my point of view. I just wrote whats coming and I tried to solve it but I cant find any starting point.
I am just making pages and assigning them frames as identity mapping AS the first_frame() which I used down just returns linear addresses and increments them everytime I call them if paging isn't enabled before. Now if GRUB memory map says I can use memory till 3 gb, then why cant I write to memory at 1102... memory? means I just did a very simple algorithm:
make a kernel directory
make pages and page tables
assign the page tables frames and keep incrementing frames everytime-> 0 page has 0x00 address frame and so on
switch to kernel directory
enable paging.
Now here I am not doing anything different for first 1 gb or beyond that. then why I get error on 1.05 gb or so? That's the question :/
And I WANT TO ASK U 1thing, If I MADE a page (blank page, just page_t* page). and add the frame say 0xa1200000(and Assume, maximum memory available to system is 0xa0000000; 0xa1200000 isn't in memory) to it BEFORE EVEN ENABLING PAGING, and then I ENABLE PAGING. Then I access the page I made before, should I get a page Fault or Should the system Crash? because I get page fault in this situation :/

Here is How I am allocating the pages :

void alloc_frame(page_t *page, int is_kernel, int is_writeable)
{
if (page->frame != 0)
{
return;
}
else
{
u32int idx = first_frame();
if (idx == (u32int)-1)
{
printf("No space left\n");
// PANIC! no free frames!!
}
set_frame(idx*0x1000);
page->present = 1;
page->rw = (is_writeable==1)?1:0;
page->user = (is_kernel==1)?0:1;
page->frame = idx;
}
}

page_t *get_page(u32int address, int make, page_directory_t *dir)
{
// Turn the address into an index.
address /= 0x1000;
// Find the page table containing this address.
u32int table_idx = address / 1024;

if (dir->tables[table_idx]) // If this table is already assigned
{
return &dir->tables[table_idx]->pages[address%1024];
}
else if(make)
{
u32int tmp;
dir->tables[table_idx] = (page_table_t*)kmalloc_ap(sizeof(page_table_t), &tmp);
memset(dir->tables[table_idx], 0, 0x1000);
dir->tablesPhysical[table_idx] = tmp | 0x7; // PRESENT, RW, US.
return &dir->tables[table_idx]->pages[address%1024];
}
else
{
return 0;
}
}


maxmem=mboot_ptr->memupper

[ashishkumar4]

I think I got a clue. The function I made thinking it does identity mapping (first_frame()) seems to have some problem for few cases :/ gonna check with different methods. if problem still persists, would edit this repy

[shmx]

Try to debug the algorithm in user space. Check the contents of the directory pages under a debugger.

[ashishkumar4]

Thanks for the advice and everyone else. I debugged it and found that the problem was really in the first_frame() function which was a part of the bitset algorithm that I earlier got from James M tutorials. I was going to re make paging algorithms but had left this part untouched. I just realized that it had several bugs including this one. I just corrected it and now its working FINE Now am gonna rip apart every James M algorithm from my kernel :3 I don't trust them now :3 Thanks everyone now, Gonna make new Virtual and physical mem allocation system

[shmx]

It is better to write new code in user space. Better to use unit tests. Only after the unit testing, you can copy the code in the kernel.