Good Kernel Debug API

[OSwhatever]

I'm trying to figure out a good general purpose kernel debugging API, an API where you can obtain statistics for any given kernel object that can be debugged. Linux does has this ability with sysfs, reading a set of files gives you info about different things in the kernel. Also many other kernels have this ability, Windows CE KITL is another example.

I would unlike Linux rather have a binary interface where the kernel returns binary data filled structs as it is simpler to transmit this through an interface for real time debugging on a desktop.

A simple example would be obtaining the memory regions for any given process. Here you have an example where you want obtain an object within another object and you also don't really know how many memory regions (objects) there are to begin with. You could have an interface for each object in the kernel but that would lead to vast amount of kernel functions. Basically how is a good way to solve this transition? Should you give a path like "/ProcessInfo/ProcessName/MemoryRegion/0"?. Then in this example you don't really know how many regions there are though which also can change if you would know it. Basically I want a good general purpose kernel debug API.

Something like: GetKernelDebugInfo("/ProcessInfo/ProcessName/MemoryRegion/0", &StructToFill);

Would a name path be too costly to resolve in real time?

I know there are some kernels that basically feeds debugging info in real time, address space changes, message traffic, memory stats. You name it, whatever you enable. Maybe real time stats should be pushed using asynch messages from the kernel to the debugging process, that either prints the info or sends the info to a desktop.

Do you have any experience with this type of debugging and what do you think is a good way to solve this.

[rdos]

I have solved this in two ways:

* I have a kernel-process that can debug anything. This process is always accessible with CTL-F8. You can see registers, modify registers, view & modify memory, single-step, run and pace. You have no access to source-code. There is also an IPC-interface that can do the same thing from another PC.

* I have integrated kernel-mode debugging into the application debugger. This was done by modifying code in the OpenWatcom project, providing special functionality in the trap-files for RDOS (even though I usually use a special app (tcpwd) that is part of RDOS as the server instead of OpenWatcom's servers that cannot handle more than one connection at a time).

You should probably start by studying other designs, and then defining your interface and how to represent threads that are debugged, as this somehow might need to be integrated into the scheduler. In my design, a thread is put in a special debug-list when it hits breakpoints or exceptions. This is how the kernel debugger can determine which threads are debugged. Other designs might freeze the whole system as part of debugging. This is a design choice you need to make. An extra bonus for embedded systems with the debug-list approach is that a special driver can be installed that listens for threads hitting exceptions, and doing automatic reboots in these situations, potentially saving fault-causes as well.

[turdus]

... Linux does has this ability with sysfs, reading a set of files gives you info about different things in the kernel...I would unlike Linux rather have a binary interface...

I don't understand. Sysfs IS a binary interface. Procfs isn't. What do you mean?
I think the best API is using files (open,read,write,close), it's easy to implement (on both sides) and extendable.

[OSwhatever]

... Linux does has this ability with sysfs, reading a set of files gives you info about different things in the kernel...I would unlike Linux rather have a binary interface...

I don't understand. Sysfs IS a binary interface. Procfs isn't. What do you mean?
I think the best API is using files (open,read,write,close), it's easy to implement (on both sides) and extendable.

True, but many standard kernel debug files do emit normal text rather than a binary format. You can of course add your own sysfs files in the Linux kernel so that they emit whatever debug format you want.

What I'm implementing is more a micro kernel like kernel and it has no concept what a file system is so exporting debug info through files in this case will not work for me.

[rdos]

A kernel debugger that requires a functional file-system doesn't sound especially useful to me.

[turdus]

A kernel debugger that requires a functional file-system doesn't sound especially useful to me.

A kernel that does not require a functional file-system doesn't sound especially useful to me

Seriously, the question is, what kind of debugger the OP wants. If it's needed to develop the kernel, fs is not a best solution. If it's needed to debug an almost ready kernel, or to debug a functional but buggy kernel fs comes into play, imho.

[OSwhatever]

A kernel debugger that requires a functional file-system doesn't sound especially useful to me.

A kernel that does not require a functional file-system doesn't sound especially useful to me

Seriously, the question is, what kind of debugger the OP wants. If it's needed to develop the kernel, fs is not a best solution. If it's needed to debug an almost ready kernel, or to debug a functional but buggy kernel fs comes into play, imho.

Microkernels usually have the file system outside the kernel as a separate server, however that's another discussion.

It's purpose is both, to debug the kernel as early in the development stage as possible well as standard debugging. Also this kind of functionality will be a method to debug user programs as well, where you can get information about resource usage for a specific process. Also message passing and task switching is interesting here. As many functions are implemented as user programs in a microkernel, this functionality is essential.

[TylerH]

You could implement it, at least at the base level, as system calls. You could do so with nonstandardized system call(s) (without guaranteeing it won't change) and implement a userland server that uses the system call to present the debug info to other processes using whatever sort of higher level interfaces(files, messages, standardized system calls, etc.) you prefer.

That's how I would do it, to satisfy my systems abstraction OCD.

[OSwhatever]

You could implement it, at least at the base level, as system calls. You could do so with nonstandardized system call(s) (without guaranteeing it won't change) and implement a userland server that uses the system call to present the debug info to other processes using whatever sort of higher level interfaces(files, messages, standardized system calls, etc.) you prefer.

That's how I would do it, to satisfy my systems abstraction OCD.

Yes, that's may plan, basically simple API functions that request info in some raw binary format.

Instead of a path, I'm thinking about just having a vector that can be used for finding the correct information item.

for example:

uint binPATH[4];

binPATH[0] = KERNELDEBUG_PROCESS;
binPATH[1] = processId;
binPATH[2] = KERNELDEBUG_GET_PROCESS_INFO;
binPATH[3] = KERNELDEBUG_END_LIST;

GetKernelDebugInfo(binPATH, ......);

I want my kernel to be as string less as possible. Just a minimal set of resources are associated with a name, therefore resolving using identifier vectors might be more convenient.

Getting the data can be done by filling a structure but I'm thinking about if passing the information through async messages instead. The advantage of doing this is that messages can have an arbitrary size when the size of structure is unknown. You can create batch jobs, for example you can request info about all processes then one message for each process can be sent automatically. Messages can be sent when a state has been changed or in given intervals. Subscribing to certain events can be subscribed.