OSDev.org

Posted: **Sat Jun 18, 2011 5:00 pm**

Okay, I don't want to sound like someone who just has other people fix their code (third post in like 3 days lol), so if someone could point me in the direction of WHY this would happen so I could figure out how to fix it, that would be great (besides, where's the fun if someone else does all the challenging parts

).

Problem: My values aren't being set in my heaps memory location (0xC0000000-0xCFFFF000)! If I try and do this (out of desperation, I tried this just after enabling paging, and mapping the kernel heap which is mapped to 0xC0000000)

(*((u32*)(0xC0000000))) = 0xBADF00D;

and then broke right after that line, and check the value in GDB and the value was still "0x0"! I am really confused at how this could happen. I have RW bits set on both the Page Directory Entry and Page Table Entry, so there shouldn't be a problem there.

This is causing my heap's hole array to contain null pointers to holes, and therefore call expand (since there are no valid holes, we assume there is no more space) and then recurse back into the allocate function. Well since no values are entered into the array(but array->size is incremented, but that is held in a static variable, so its not part of the same memory) the function keeps recursing until the stack is completely screwed over!

Any ideas?

Thanks, Caleb.

Posted: **Sat Jun 18, 2011 7:06 pm**

Okay, I went back and tried this:

Code: Select all

u32 phys = vmm_get_phys(0xC0000000, kdir); // Get the physical address of the start of the heap
vmm_map(phys, phys); // Identity map it
*((u32*)phys) = 0xBADF00D; // Set it's value
k_printf("\nphys=0x%x", *((u32*)phys)); // Print it's value

and it worked. The value was indeed set to "BADF00D". This means that it is a problem with my page mapping. I will have to go over my mapping code, but thought I would post this to see if it gave anyone an idea. This makes the error even wierder, because this means there is a problem with my page mapping, and yet I'm not getting a Page Fault... hmmm....

EDIT

It appears that this is a problem with VMs. I decided to run it from a flash drive, and it works. I don't get my "Assertion failed... etc" message (which was stating that a variables value was wrong. I used an assertion since I don't have the debugger on a physical machine btw). It doesn't work with VirtualBox or qemu. Bochs is a pain to setup so I haven't tried it (well actually it just didn't like my disk image because of the geometry crap).

EDIT 2

Just confirmed that it is working with the physical machine. I ran this:

Code: Select all

	k_printf("\tMemory test:\n\tSetting *((u32*)(0x%x))=0x%x\n", KHEAP_START, 0xBADF00D);
	*((u32*)(KHEAP_START)) = 0xBADF00D;
	k_printf("\t*((u32*)(0x%x))==0x%x\n",KHEAP_START, *((u32*)(KHEAP_START)));

Just after enabling pagin, and it worked correctly. I don't understand how this could not work. I know that mainstream OS's use higher-half kernels which need access to addresses like 0xC0000000, and THEY run in qemu/VB, so it must be something with my settings in the VM or my code. I think it's the VM settings since my code works on a physical machine.

EDIT 3

It appears the VMs aren't using the page tables correctly, because when I set my heap address to 0x70000000 and give my VM an outragous amount of memory ( i think I slid the bar in VB to like 2.9 GBs) it runs. The amount of physical memory that the machine has should not matter though! Paging is enabled and the memory is mapped to a address within the current memory size! This doesn't make sense...

Posted: **Sun Jun 19, 2011 1:53 am**

Several points to note:
- identity mapping 0xC0000000 requires RAM to be present at that very same physical address (i.e. you have more than 3G installed, there's a video card there, or also an option in actual hardware, you are reading back bus float)
- optimisations screw up any form of memory probing, you will want them off for testing and you should use volatile where appropriate.
- The VM is never wrong until you have proof of the opposite.

Bochs is a pain to setup so I haven't tried it (well actually it just didn't like my disk image because of the geometry crap).

In that case, your disk image is broken or misconfigured, which implies a bug in your code. Broken disk images are later often the cause of hard-to-find bugs.

Posted: **Sun Jun 19, 2011 7:04 am**

1. I'm not identity mapping 0xC0000000. It is being mapped to a block allocated with my physical memory manager. I'm then (for testing) identity mapping that physical address to see if I can write there.
2. Good Idea.
3. I'm still reluctant to say that, because that doesn't make sense, but I don't see where my code causes this. I'm not sure what it is at this point. It's acting like I'm trying to access memory above what I have, but I'm not mapping anything that high. I map my heap like so:

Code: Select all

u32 virt = KHEAP_START, phys = 0;
for(virt; virt < (KHEAP_START+KHEAP_INIT_SIZE); virt += 0x1000){
     if( !(phys = pmm_alloc_block()) ) panic("No memory!");
     vmm_map(phys, virt);
}

and if I also identity map that physical I address, I can access it so that means that my heap is pointing to a valid address.

In that case, your disk image is broken or misconfigured, which implies a bug in your code. Broken disk images are later often the cause of hard-to-find bugs.

Well it's just telling me that my CHS doesn't line up with my image, but I used standard H/S (16/63 respectively) and used those plus the number of blocks in the disk to calculate C.

Posted: **Sun Jun 19, 2011 8:36 am**

Caleb1994 wrote:
In that case, your disk image is broken or misconfigured, which implies a bug in your code. Broken disk images are later often the cause of hard-to-find bugs.
Well it's just telling me that my CHS doesn't line up with my image, but I used standard H/S (16/63 respectively) and used those plus the number of blocks in the disk to calculate C.

and C * H * S * 512 = exactly the image size? I expect that the C you calculated turns out to be a fractional number.

That said, when you fix that problem, you can start using bochs debugger which has the very nice feature to list the page mappings (info tab), you can use that to verify that your code does what you expect from it, including cases where your VMM/PMM screws up and ends up using wrong addresses at some point.

Posted: **Sun Jun 19, 2011 4:28 pm**

Combuster: Yes it was a float. I did also try and take the integer form of that value and create a new image using the size reported by: (int)(OLD_C) * H * S * 512 and it still didn't like it.

Have you tried printing your page directory to see if it correct?

Not the hole thing, but I have implemented a function to take a virtual address and a directory, and print out some information. Here is the information I got from KHEAP_START (0xC0000000):

Virtual Address Debug Information, Address: 0xC0000000
Page Directory Entry: index=768; flags=p,rw,s; frame=0x8000
Page Table Entry: index=0; flags=p,rw,s; frame=0x7000
Translated Physical Address: 0x7000

It seems to be mapped correctly. I checked, and the allocated address is, in fact, 0x7000, so the frame is correct, and just in case you don't know (I took it from the intel manual, so it should be common) but p,rw,s is Present, Read/Write, and Supervisor, so this should work also.

Posted: **Sun Jun 19, 2011 9:14 pm**

Bochs is telling me paging is off when I call "info tab" but telling me it's on when calling "creg" here is the output from creg which indicates that CR0 says paging should be on.

CR0=0x60040017: pg CD NW AC wp ne ET ts EM MP PE
CR2=page fault laddr=0x0000000000000000
CR3=0x0000000000001000
PCD=page-level cache disable=0
PWT=page-level write-through=0
CR4=0x00000000: osxsave pcid fsgsbase smx vmx osxmmexcpt osfxsr pce pge mce pae pse de tsd pvi vme
EFER=0x00000000: ffxsr nxe lma lme sce

I was confused at first why bochs output from "page 0xC0000000" gave me this:

linear page 0x00000000c0000000 maps to physical page 0x00000000c0000000

but now it makes sense. It thinks paging isn't on, so it just uses all addresses as if they were identity mapped.

Do you know why bochs would tell me that paging is off but also tell me that paging is on (indicated by the "pg" flag in CR0)?

Posted: **Sun Jun 19, 2011 9:20 pm**

Code: Select all

CR0=0x60040017: pg CD NW AC wp ne ET ts EM MP PE

Doesn't the lowercase pg mean that paging is off ? Certainly bit 31 is clear.

Posted: **Sun Jun 19, 2011 11:30 pm**

Okay, well it seems that a couple of bugs that I recently fixed have shown me problems that weren't manifesting themselves before. I'm now in the process (now that I realize what is happening) of fixing these little bugs. The mapping problem stems from paging actually not being enabled, which is from another (one character) typo. And because, until the heap, I haven't had to mess with anything except identity mapped stuff, these problems haven't shown themselves. Apparently there is also a problem with my page tables, so I'm going to spend a few days (or maybe a week and a half

) finding tiny bugs/typos in my code. Is it weird that, in all honesty, I find this is the funnest part of programming? Tracking down those pesky bugs that just plain make you look dumb? xD

Posted: **Mon Jun 20, 2011 12:07 am**

Caleb1994 wrote:Is it weird that, in all honesty, I find this is the funnest part of programming? Tracking down those pesky bugs that just plain make you look dumb?

Well, it definately isn't. Struggle makes success more fun. When you dig out the root of the problem and finally discover a workaround, you feel like you conquered the world. That's what programming is all about.

Now on to the topic. I think you should play with paging a little more before you focus on Virtual Memory Manager. Just make sure that everything is mapped as you would've desired and you're ready to go..........

Good Luck.

Posted: **Mon Jun 20, 2011 6:49 am**

Caleb1994 wrote:Is it weird that, in all honesty, I find this is the funnest part of programming? Tracking down those pesky bugs that just plain make you look dumb? xD

An anecdote on the side.

I once worked several days on a particularly nasty bug, which turned out to have a downright stupid reason. I don't remember what it was, exactly, but it was an error that should have been easy to avoid, and was extremly easy to fix.

I was pretty crestfallen when I went to my technical lead to report what the error was. I felt so stupid - first for making the mistake, then for taking so long to find it.

He just shrugged and said something I remember fondly, and have quoted often since:

"Every bug is trivial... once you have found it."

Remember this when they ask you how long it will take to fix a bug. Fixing usually takes a couple of minutes. Finding the critter, that's the trick.

Just don't make a habit out of taking ages to find the little buggers. Experience helps a lot in this department.

Posted: **Mon Jun 20, 2011 9:01 am**

berkus wrote:Thanks for taking my advice.

Sure thing! If I wasn't going to take advice, why would I come to the forum

Chandra wrote:
Caleb1994 wrote:Is it weird that, in all honesty, I find this is the funnest part of programming? Tracking down those pesky bugs that just plain make you look dumb?
Well, it definately isn't. Struggle makes success more fun. When you dig out the root of the problem and finally discover a workaround, you feel like you conquered the world. That's what programming is all about.

Now on to the topic. I think you should play with paging a little more before you focus on Virtual Memory Manager. Just make sure that everything is mapped as you would've desired and you're ready to go..........

Good Luck.

1. Amen.
2. Agreed. I don't like the bochs debugger much compared to gdb, but the ability to view page tables and the control registers is useful so that will definitely help!

Solar wrote:"Every bug is trivial... once you have found it."

Love it. Definetly saving that one to "~/quotes/trivial-bugs.txt" lol

Posted: **Mon Jun 20, 2011 11:20 am**

Alright. Looks like I found my paging problems. I looked back at the PDE/PTE flags, and re-wrote my flags enum in terms of bit shifts instead of hex values to make it more readable and to make sure they were correct (so bit zero would be (1<<0), one (1<<1), two (1<<2), etc...) and completely re-wrote my initialization/mapping, and I now get this from bochs!

<bochs:3> info tab
cr3: 0x0000000000001000
0x00000000-0x00ffffff -> 0x0000000000000000-0x0000000000ffffff
0xc0000000-0xc0000fff -> 0x0000000000007000-0x0000000000007fff
0xc0001000-0xc0097fff -> 0x0000000000009000-0x000000000009ffff
0xc0098000-0xc00fffff -> 0x00000000000e8000-0x000000000014ffff

Thanks again guys for all the suggestions! I know I will probably find another bug, but at the moment, my paging code is bug free

OSDev.org

Values not being set

Values not being set

Re: Values not being set

Re: Values not being set

Re: Values not being set

Re: Values not being set

Re: Values not being set

Re: Values not being set

Re: Values not being set

Re: Values not being set

Re: Values not being set

Re: Values not being set

Re: Values not being set

Re: Values not being set