OSDev.org

Hi all!
Anybody knows the best way of protected mode programs debugging??

if your talking about your OS, write your own debugger using the i386 inbuilt debugging stuff (DRx registers), etc.

if your talking about pmode debugging in windows or something, try soft-ice etc

I?v start to write my own os with DJGPP C/C++. How to debug received code? To write my own debugger for c/c++ is too difficult

Debugging is a difficult task. ?Writing a debugger sounds like an even more difficult task (wouldn't know, haven't done it yet, but I'm looking forward to "debugging the debugger" ). ?Here's what you COULD do:

1) Get a copy of NASM, or use the -S option under GCC
2) If you get NASM, you can use ndisasm to dissasemble the object code that was produced by GCC, this will give you code offsets for all of your instructions, and if you know where you've loaded your OS and where this particular module fits in, you can calculate what the linear address of the code will be
3) If you use the -S option, GCC will generate the assembly but will not assemble it, you can then look at this code (WARNING: It's in AT&T syntax!). ?This won't generate the listing file for you (does anybody know how to do that with gas?)
4) Anyway, you can plunk down some money for VMWare (or just get the temp license). ?In the options, turn on extra debugging log info.
5) Run your OS inside VMWare, if you get an unhandled GP# or any other unhandled exception, VMWare will most likely crash. ?
6) Once this happens, check inside your VMWare/WM/MyOS directory and look for VMWare, VMWare.txt, VMWare.log, something like that, and open it
7) Scroll down to the bottom, you'll see the last instruction's offset that caused the fault plus the type of fault and error code. ?You can then use this information to narrow down what went wrong

Of course, once you install your exception handlers, VMWare will let your OS handle exceptions and won't crash unless the virtual machine manages to triple-fault, so this technique only works for the early stages of OS design. ?Also, this won't help with more subtle bugs that don't generate exceptions. ?I haven't gotten to that point yet so I won't pretend to give you advice, but I'll probably be writing my own debugger (in the simplest sense of the word) pretty soon now. ?But it will be difficult, and will probably take me a while to figure out how to write one. ?

I'm not trying to be rude or anything, but if you don't want to write a piece of code because you think it's too difficult, then why do you want to write an operating system?

Hi!
If we are talking about simple debugger then i can say - im using bochs pc emulator (http://bochs.sourceforge.net) with integrated debugger instead of VMWare.
I talked about complicated c/c++ debugger - i dont know how i can do it for my os if i writing it in c/c++.

Some techinque of mine : embed the debug in your code
Here's what a Clicker function would look like

my_fct( ... )
{
enter(DBMISC,"my_function");

sign("allocating memory ...");
mem=kalloc(memsize);
sign("%x\n",mem);

sign("wiping ...");
memset(mem, 0, memsize);
sign("done\n");

leave();
}

And when the system will be running, it'll show
[entering my_function]
allocating memory ... 12345678
wiping... done
[my_function done]

or
[entering my_function]
allocating memory ... 00000000
wiping ...
*PAGE FAULT EXCEPTION DETECTED*
* SYSTEM HALTED *

Well, you don't necessarily have to put *that* much debugging informations in your code: just pour it with your favourite mood.

http://clicker.sourceforge.net/docs/cake/debug.html

Do i need to say that enter(...), sign(...) and leave() are automatically left non-written if you don't define __DEBUG__ at compile time ? so you can have a kernel that is fast and small (and virtually impossible to debug) or a kernel that is nicely debuggable one

Other common tricks in embedded debugging:
assert(condition, message) : will halt the system if condition isn't met
anykey() : prompt the developer for a keypress before going further (step-by-step kernel execution with coder-defined breakpoints).

This is very valuable especially if you're able to view these informations just before the system reboots (i.e. have them somewhere on screen or sent through network to a remote debugger ... whatever

The debugger built into Bochs is VERY good and that is what I'm using.

You can put a breakpoint in gcc/DJGPP code like this:
[tt]
asm("cli");
asm("hlt");
[/tt]

or in assembly code:
[tt]
cli
hlt
[/tt]

This is useful as the bochs "out file" will tell you if it encountered the [tt]hlt[/tt] instruction(usefull for finding out how far your code can go before it crashes).

K.J.