Implementing a kernel debugger

[Colonel Kernel]

I'm wondering if anyone here has implemented a kernel debugger for their OS -- especially if your OS is based on a microkernel architecture. I'm thinking especially of remote kernel debuggers that work via a serial connection.

What I'm wondering specifically is: How much of the kernel debugger has to be in the microkernel itself? Is it like a big lump of code in the kernel itself that gets enabled only in debug builds? Or does it include some stuff outside the kernel as well (other than the client app on the other end of the serial link, if any)?

[oswizard]

Answer: It depends.

What I have done is written a very basic debugging interface in kernel mode. Now, I use a monolithic kernel, but I imagine a debugging interface ought to be in the central unit of a microkernel, as this would facilitate debugging of the message-passing/whatever-you-want-to-call-it architecture.

My kernel debugger writes directly to the serial port and connects to a program I wrote on WinXP. The debugger simply waits for interrupts: int 1 or int 3 for debug breakpoints, or the serial interrupt for data arrival. Basically all it does is send the CPU context (regs, crx, drx, gdt, idt, etc.) to the debugger, and the debugger can send read or write requests to the OS. With this I was able to implement a debugger much like BOCHS's debugger, but I hope to expand on it later.

Anyway, my advice: keep it simple. The debugger should be able to function with a minimum of overhead to allow the greatest amount of code to be debugged. Also, a simpler debugger means it is less likely to contain bugs in itself.

A final word of warning: watch out and do not set breakpoints in code that the debugger calls, such as serial-io routines, or the main interrupt handler. A particularly nasty infinite loop or triple fault will result.

Good luck,
Mike

[Colonel Kernel]

Thanks for the info!