Symbols in the kernel

[kscguru]

I've finally got a nice multistage bootloader so I can bring up the 64-bit environment I want for my kernel. And going through all the ELF segments got me thinking ...

What is the role of kernel symbols?

I have a few thoughts:
1) Debugging - panic() could walk a backtrace and print function names, which makes my development life happier.
2) Loadable modules
2a) Linux approach: kernel stores symbols does all dynamic linking
2b) MacOS X approach: privileged userspace app mmaps kernel, performs linking with symbol table in userspace.
3) Passing symbol tables between bootloader stages gets to be a pain fast.
3a) The next-to-last stage could copy symbols somewhere, or
3b) Symbols could come from a file on the filesystem (but I don't have a filesystem yet...)
4) Symbol table parsing => string parsing => insecure C code

Opinions? Should symbols live in the kernel? Should symbols come from the boot image or from something loaded after boot?

[Combuster]

string parsing => insecure C code

Really? where do you base that on?

[JamesM]

string parsing => insecure C code

I think possibly you're confusing "strcpy() considered harmful" with "string parsing in general considered harmful". Just use strncpy and all strn-related functions and you'll be fine.

[kscguru]

Just use strncpy and all strn-related functions and you'll be fine.

Even strncpy isn't safe enough for my paranoia - it doesn't guarantee NUL-termination. (Some systems have strlcpy, which is better.)

Code: Select all

void foo() {
   char *buf = malloc(PAGE_SIZE);
   char *src = "PAGE_SIZE bytes"; // PAGE_SIZE+1 bytes total
   strncpy(buf, src, PAGE_SIZE);
   printf("%s", buf);   // #PF as this reads past end of buf
}

Sure I could use a non-standard strncpy that does terminate, but this is a digression - let's just say I have a design goal of doing as little string processing as possible, especially in a kernel.

Back to symbols please ...

[ucosty]

Even strncpy isn't safe enough for my paranoia - it doesn't guarantee NUL-termination. (Some systems have strlcpy, which is better.)

You're not limited to stdlib functions. Write your own secure string functions, it can't be that hard.

In any case cant you just load and parse your symbols table once and then just pass around references through the stages?

[JamesM]

Imho you're setting yourself an arbitrary and impossible goal by avoiding all string handling... My strncpy function *does* add a null terminator.

[Brynet-Inc]

string parsing => insecure C code

Why don't you just implement strlcpy/strlcat.. snprintf and friends?

Most C libraries have an implementation these days, I'm sure you'll find one with a compatible licence.

http://www.usenix.org/events/usenix99/millert.html
http://en.wikipedia.org/wiki/Strlcpy

[kscguru]

Folks ... I asked for thoughts about where good places to store and process symbol tables are. I see zero posts about symbols and five posts quibbling about my perception of the safety of string processing code.

Yes, I can write safe string processing code; it takes careful coding and deviates from language standards, so I would prefer to not do so in my kernel, but I can.

Where should symbol table processing go, and what tradeoffs does everyone see in the different approaches?

[bewing]

During development, for debugging purposes, a symbol table can be vaguely useful. But IMHO, a finalized kernel should not have a symbol table at all. You do not want people tracing into your kernel. You do not want to have to recompile all your apps against the kernel, every time the kernel changes slightly. It's annoying to have to link all your apps against the kernel "library" every time you recompile an app. A symbol table takes a great deal of space -- making your kernel harder for the bootloader to load, taking more time, wasting memory space.

I think it's best just to export entrypoints in a single table. The table never mutates, so nothing ever needs to be recompiled. You can pass the pointer to the table around.

[Brynet-Inc]

My strncpy function *does* add a null terminator.

Hopefully it does so in a standards compliant way... changing defined behaviour doesn't sound very wise.

[ucosty]

As far as I know Windows developers use a tool (Windbg) to create symbolised backtraces from more general debugging dumps generated by the OS.

The linux kernel does do kernel backtraces if it (or a drive module) crashes.

What is the role of kernel symbols?

They simplify debugging by allowing you to map instruciton addresses to their containing functions. This lets you, for instance, let you walk through a stack backtrace without having to manually convert the EIP addresses to functions. This is useful because in a lot of cases the cause of a problem is not in the function that crashed but the calling function.

Opinions? Should symbols live in the kernel? Should symbols come from the boot image or from something loaded after boot?

Symbols can live anywhere you want. Mine, for instance, are loaded by grub as a module and the kernel gets direct access to them. This will change in the future when I make them loadable through my filesystem interface.

The earlier you load your symbol table the faster you can use it (obviously).

During development, for debugging purposes, a symbol table can be vaguely useful. But IMHO, a finalized kernel should not have a symbol table at all.

It's all about making tedious work lighter. If you prefer to do manual offline translations then so be it. I dont agree about the final kernel not containing symbol tables.

You do not want people tracing into your kernel.

Why not? They have access to the source code. Why make it harded than necessary debug an OS/drive problem. Take a normal end user scenario where a buggy driver crashes the kernel. The backtrace logs along with other debugging dump information can be provided to the developers to help them fix the problem. It simplifies providing feedback and allows people to gain insight into where the problem lies. It could, for instance, be a bug in an edge case in a specific syscall.

You do not want to have to recompile all your apps against the kernel, every time the kernel changes slightly.

Why would you have to do that?

It's annoying to have to link all your apps against the kernel "library" every time you recompile an app.

Again why would you have to do that? A kernel symbol table only concerns the kernel. If you have an external symbol table loaded from disk than any programs (such as debuggers) that need that information can parse it themselves like Windbg - from disk.

deviates from language standard

Within your kernel you have no standard libraries and no obligation to provide them. Userspace applications whose developers may wish to use stdlib functions would use a user space standard library. This is completely seperate as I'm sure you are aware. If you are worried about standard library code not covering your requirements and you are working in a completely standalone environment (i.e your kernel) you don't have to follow those standards. So long as you are consitent within your kernel to save yourself some frustration.

[JamesM]

My strncpy function *does* add a null terminator.

Hopefully it does so in a standards compliant way... changing defined behaviour doesn't sound very wise.

It's in *my* kernel. So *I* define the expected behaviour.

[pcmattman]

You do not want to have to recompile all your apps against the kernel, every time the kernel changes slightly.

A little thing called dynamic linking - you don't need to recompile .

[Brynet-Inc]

My strncpy function *does* add a null terminator.

Hopefully it does so in a standards compliant way... changing defined behaviour doesn't sound very wise.

It's in *my* kernel. So *I* define the expected behaviour.

Wouldn't it be better to give the function a different name then?

[kscguru]

As far as I know Windows developers use a tool (Windbg) to create symbolised backtraces from more general debugging dumps generated by the OS.

Core dumping is definitely useful - but in the realm of hobbyist OS, it's so very easy to attach the debugger. (I dare say core file formatting is more of a black art than dynamic linking!)

That's a very useful thought ... in-kernel symbols are most useful when attaching that debugger is difficult. Which does make me less worried about having them immediately.

Symbols can live anywhere you want. Mine, for instance, are loaded by grub as a module and the kernel gets direct access to them. This will change in the future when I make them loadable through my filesystem interface.

Grub, alas, cannot load mine; there is a 32-to-64-bit switch, so I have to load myself in a bootstrap stage. (Not terribly hard to do.)

The principle usage I have is for module loading. But I also like bewing's suggestion of an export table - much easier than linking a whole ELF image (relocations and all). Or maybe even a hybrid - export-table for boot-time modules (e.g. the hard disk containing symbols), something fancier once I can pull in symbols from disk.

Ideas are fun. Now to go implement...