System crashes when trying to boot on a real computer

[ilmmatias]

Hello, so, my OS was working perfectly (as far as i tested) in VirtualBox and VMWare, but when i tried to boot it in my real computer (my OS already have SATA support, so it was supposed to work), i got a page fault, i got to trace it to when the SATA driver was trying to allocate some memory to initialize one of the devices, i don't have any idea of why this is happening, can someone help me?

[bzt]

Hi,

Try out your OS with qemu and bochs. If they are producing your page fault problem, then you can use

Code: Select all

qemu -s -S

and connect gdb to it, or use the bochs debugger to find your bug.

Just a sidenote, VirtualBox also has a quite nice built-in debugger, but you have to start it with

Code: Select all

VirtualBox --debug

or

Code: Select all

VBOX_GUI_DBG_ENABLED=true VirtualBox

to make "Debug" option appear in the menu. But if the problem does not occur in VB, then probably this won't do any good for you. Btw, have you checked the vm's log? VB log can be extremely helpful.

Cheers,
bzt

[ilmmatias]

Hi,

Try out your OS with qemu and bochs. If they are producing your page fault problem, then you can use

Code: Select all

qemu -s -S

and connect gdb to it, or use the bochs debugger to find your bug.

Just a sidenote, VirtualBox also has a quite nice built-in debugger, but you have to start it with

Code: Select all

VirtualBox --debug

or

Code: Select all

VBOX_GUI_DBG_ENABLED=true VirtualBox

to make "Debug" option appear in the menu. But if the problem does not occur in VB, then probably this won't do any good for you. Btw, have you checked the vm's log? VB log can be extremely helpful.

Cheers,
bzt

With QEMU it works just like VirtualBox and VMWare, and Bochs doesn't have UEFI support so i can't test it using Bochs.
And my OS just doesn't works in my physical computer, if i try in QEMU, VirtualBox or VMWare it works normally.

[bzt]

With QEMU it works just like VirtualBox and VMWare, and Bochs doesn't have UEFI support so i can't test it using Bochs.
And my OS just doesn't works in my physical computer, if i try in QEMU, VirtualBox or VMWare it works normally.

Hmmm, not good. But I can't give you other advice than to debug and try to figure out why the page fault is happening. If I were you I would start at EFI_GET_MEMORY_MAP. Unlike virtual machines, EFI on real computer tend to generate hundreds of small entries. Could be too many or could be one that's smaller than a page, or overlapping entries, something like that. Inproper input to your allocator could very well cause your page fault issue.

Cheers,
Z

[Octocontrabass]

Bochs doesn't have UEFI support so i can't test it using Bochs.

Yes it does. Check out this thread for details on setting it up.

[Schol-R-LEA]

@CHOSteam: In the latest update to your repo, the comment is simply, "Oops, how i could let this go?", which doesn't really indicate what the problem you found was. The changes themselves were notable (corrections to three mis-coded comparisons in the memory management, and adding a return statement in a process creation function - IOW, significant mistakes, but the sort anyone could make, and not really related to the logic of the code), but since it was entered just before your last post here, I assume it wasn't the root of the problem you're discussing. Would you mind describing it in more detail, so we have a clearer idea of what is going on behind the scenes?

At this point, it is hard to say what would be relevant, so any additional data is (up to a point) useful. Please elaborate on both how you installed the OS to the drive, and how your virtualizing environments are configured. Perhaps a white paper added to your git repo's documentation section?

It might help if we had a clearer idea of the hardware setup - the processor, memory, and specifically the SATA drive. In particular, it would help to know if a) the drive an external one, and if so, is it attached as eSATA, b) this is being tested on same hardware as the development system you run the virtualizers on, or a different, dedicated test rig, and c) you are using the same drive in the virtual environments (as opposed to having a virtual file image on the development system).

Finally, since this really sounds more like a memory management issue than a SATA driver issue, you might want to write a dummy/test driver which only makes memory allocation requests, and see if it causes the same sort of crash. This should help isolate the problem.

[ilmmatias]

With QEMU it works just like VirtualBox and VMWare, and Bochs doesn't have UEFI support so i can't test it using Bochs.
And my OS just doesn't works in my physical computer, if i try in QEMU, VirtualBox or VMWare it works normally.

Hmmm, not good. But I can't give you other advice than to debug and try to figure out why the page fault is happening. If I were you I would start at EFI_GET_MEMORY_MAP. Unlike virtual machines, EFI on real computer tend to generate hundreds of small entries. Could be too many or could be one that's smaller than a page, or overlapping entries, something like that. Inproper input to your allocator could very well cause your page fault issue.

Cheers,
Z

Ok, i got to make my OS dump the received memory map, here is the memory map that it is getting on my physical machine: https://i.imgur.com/SVEiohx.jpg

[ilmmatias]

@CHOSteam: In the latest update to your repo, the comment is simply, "Oops, how i could let this go?", which doesn't really indicate what the problem you found was. The changes themselves were notable (corrections to three mis-coded comparisons in the memory management, and adding a return statement in a process creation function - IOW, significant mistakes, but the sort anyone could make, and not really related to the logic of the code), but since it was entered just before your last post here, I assume it wasn't the root of the problem you're discussing. Would you mind describing it in more detail, so we have a clearer idea of what is going on behind the scenes?

At this point, it is hard to say what would be relevant, so any additional data is (up to a point) useful. Please elaborate on both how you installed the OS to the drive, and how your virtualizing environments are configured. Perhaps a white paper added to your git repo's documentation section?

It might help if we had a clearer idea of the hardware setup - the processor, memory, and specifically the SATA drive. In particular, it would help to know if a) the drive an external one, and if so, is it attached as eSATA, b) this is being tested on same hardware as the development system you run the virtualizers on, or a different, dedicated test rig, and c) you are using the same drive in the virtual environments (as opposed to having a virtual file image on the development system).

Finally, since this really sounds more like a memory management issue than a SATA driver issue, you might want to write a dummy/test driver which only makes memory allocation requests, and see if it causes the same sort of crash. This should help isolate the problem.

The crash happens just after one of the MemAllocate calls in the SATA driver, i'm also thinking that probably the problem is inside of the allocator.
The drive isn't an external one.
Yes, i'm testing in the same hardware that the virtualizers are running, as this is the only computer that i have for now.
I tried using the physical CD using VirtualBox (attaching it instead of the ISO image) and it worked normally.

EDIT: I tried unit testing the allocator and it triple faulted when trying to allocate 10000 entries each one with the size of one Int, it crashed when i was equal to 5689.

[bzt]

Hi,

Ok, i got to make my OS dump the received memory map, here is the memory map that it is getting on my physical machine: https://i.imgur.com/SVEiohx.jpg

I'm sorry, imgur.com does not pass my firewall. But the point is, make sure your free memory list in the allocator is not corrupted for sure. I for example convert the system memory map into a list which contains only the free segments, and I make sure of it that there're no overlaping segments, and also that all pages are excluded from the converted list which are listed as non-free. (So for example if I have an entry like non-free base 0x1020 size 128 bytes then I exclude the whole page 0x1000-0x1FFF from the list to be sure. My pmm allocates only pages, but system memory map on real hardware could contain smaller blocks.)

EDIT: I tried unit testing the allocator and it triple faulted when trying to allocate 10000 entries each one with the size of one Int, it crashed when i was equal to 5689.

That's good (in a way). There should be nothing hardware specific in the allocator, meaning you can compile it in a normal Linux executable and run through valgrind. That will help you to find the problem.

Cheers,
bzt

[Octocontrabass]

Ok, i got to make my OS dump the received memory map, here is the memory map that it is getting on my physical machine: https://i.imgur.com/SVEiohx.jpg

I see in that picture you're using a computer with RAM mapped above 4GB, but your code never uses the base_high or length_high values. You need to check the whole 64-bit addresses in the memory map when determining what RAM is available for your OS to use. (A 32-bit OS can still access all physical memory using PAE.)

[ilmmatias]

Hi,

Ok, i got to make my OS dump the received memory map, here is the memory map that it is getting on my physical machine: https://i.imgur.com/SVEiohx.jpg

I'm sorry, imgur.com does not pass my firewall. But the point is, make sure your free memory list in the allocator is not corrupted for sure. I for example convert the system memory map into a list which contains only the free segments, and I make sure of it that there're no overlaping segments, and also that all pages are excluded from the converted list which are listed as non-free. (So for example if I have an entry like non-free base 0x1020 size 128 bytes then I exclude the whole page 0x1000-0x1FFF from the list to be sure. My pmm allocates only pages, but system memory map on real hardware could contain smaller blocks.)

EDIT: I tried unit testing the allocator and it triple faulted when trying to allocate 10000 entries each one with the size of one Int, it crashed when i was equal to 5689.

That's good (in a way). There should be nothing hardware specific in the allocator, meaning you can compile it in a normal Linux executable and run through valgrind. That will help you to find the problem.

Cheers,
bzt

I tried testing the allocator on linux, and it worked normally (i made 5 tests with function to allocate over 1mb of memory and allocate lots of small blocks), it just doesn't works on my OS.

[bzt]

I tried testing the allocator on linux, and it worked normally (i made 5 tests with function to allocate over 1mb of memory and allocate lots of small blocks), it just doesn't works on my OS.

That's isn't much. Try more, test all the edge cases, so that every conditional block of your allocator gets called at least once in at least one of the tests. And run through valgrind, and don't stop until it says everything is ok. I really hope this helps. If that works okay, then you can move on to check the OS interfaces that your allocator uses, because if the problem is not in your allocator then it must be in one of the underlying function.

Good luck,
bzt

[ilmmatias]

After doing more tests with the allocator, I discovered that I was using too much stack space in the test function, I increased and now the test function works.
But the kernel still crashes when calling the MemAllocate from my AHCIInit function, and this only happens in my physical computer.
I tried to remove everything that I could from the kernel, so I could try to isolate the problem, still, I don't have any idea of where it is. Looks like something is corrupting the allocator blocks, and it isn't my tasking implementation, as the AHCIInit funcion is called before it. (I attached this minimum kernel, I just removed the font file as it was making the zip file too big)

[MichaelPetch]

your kernel zip file doesn't seem to contain font_psf.oo

[ilmmatias]

your kernel zip file doesn't seem to contain font_psf.oo

I know, I said that I haven't included font.psf as it was too big for the uploader, you can download it here: https://github.com/CHOSTeam/CHicago-Ker ... r/font.psf