libc in kernel space

[bluemoon]

I'm current porting newlib to my kernel, and can't wait to adopt it everywhere, including kernel and drivers.

However, there are major difference for kernel and user land application, for example sbrk.
For example, user land application calling sbrk will expand the address space according to current heap located at process information structure, however for kernel service handlers and drivers, calling sbrk will cause trouble since they do not own any process - they need ksbrk / kmalloc which works on kernel's own heap.

On there other hand, newlib provide many convenient tool which interested me a lot.

So, is there some well know subset of the newlib which has no dependency function that give you surprise ?
Or I should put the effort to unify the environment, eg switch heaps, so that in kernel space I can use newlib safely ?

[OSwhatever]

I use newlib (in practice it could be any clib) but in a very limited way. Basically only functions from string.h and other "romable" functions. No functions that use file access or memory allocations are allowed. If any function ends up using clib memory allocations you usually crash and if you have file operations the linker will complain. You usually write your own file handling and memory allocation in the kernel anyway. clib memory allocations are not always suitable in kernel mode.

When you get out to user space, you can use a lot more of the clib.

[bluemoon]

Thanks, that's exactly my concern.

Currently I removed my own memcpy/strcpy etc and used newlib version; seems I should stop there and not proceed further.

No functions that use file access or memory allocations are allowed

Sometime it's hard to tell if the function used file or memory allocation, for example I know printf used file, but not expect it calling sbrk and malloc. I am simply not so sure for every functions to bet on it.

[Solar]

Hm, strange... I managed to implement a printf() that doesn't need memory allocations, and it wasn't really difficult... OK, I don't do wide chars or floating point yet, but I don't think there's anything in there that would require dynamic memory, either.

[bluemoon]

yeah that was totally out of my expectation, newlib printf used sbrk.
When I have this on my kernel:

Code: Select all

printf ( "Hello %d\n", 123456 );

There following syscall is detected:

Code: Select all

fstat ( 1, buf ) -> kernel return fail
sbrk ( 0x418 ) -> kernel return NULL
sbrk ( 0xBE8 ) -> kernel return NULL
write ( 1, 8, 13 )

Well, my kprintf don't need memory allocation too.

[gerryg400]

It's actually quite odd that printf needs malloc. It really annoys me because I had to write my own kmalloc driver. Anyway, I use newlib in my kernel and in userspace. My kernel is long mode so I build 2 copies of newlib (one -mcmodel=kernel the other -mcmodel=large).

In the microkernel itself I stick to the standard mem*, str* but also use time functions. The newlib documentation gives the dependencies for every function and after a short time you remember what you can and can't do.

Because I link my kernel against a full newlib, if I were to accidentally call malloc and printf I wouldn't get a link error. Actually I'm not sure what would happen really I don't think I've ever done it. I would expect that malloc would work but that printf would cause a fault.

[Solar]

yeah that was totally out of my expectation, newlib printf used sbrk.
When I have this on my kernel:

Code: Select all

printf ( "Hello %d\n", 123456 );

There following syscall is detected:

Code: Select all

fstat ( 1, buf ) -> kernel return fail
sbrk ( 0x418 ) -> kernel return NULL
sbrk ( 0xBE8 ) -> kernel return NULL
write ( 1, 8, 13 )

Apparently newlib checks the status of stdout (fstat()), finds the handle invalid (why? errno could perhaps tell you), and then - I assume - tries to open it. I.e., it's not printf() doing the sbrk(), it's fopen() (which is somewhat understandable but unsatisfactory).

You should find out why the fstat() call fails. Is stdout (fd 1) open and OK to write to? I'd suspect no, which would mean that e.g. puts() or putc() should likewise fail.

PDCLib uses a static initialization of stdin / stdout / stderr, and leaves it up to the host system to set up it's side of things appropriately. Yes, checking if things really are good like newlib does with fstat() adds security, but you just got bitten by the downside of that - the automatic reaction of newlib (opening the stream) isn't what you want, and has unwelcome side effects...

[bluemoon]

It turns out the sbrk() is called on first function call on some functions in the newlib (like malloc or printf)
I guess it's kind of library-wise initialization.

If I write the application this way:

Code: Select all

#include <stdio.h>
#include <stdlib.h>
int main(int argc, char *argv[]) {
    char * p = (char*) malloc(16);
    printf ("Hello from user application!\n");
    return 0xBADC0DE;
}

This sequence is detected (sbrk and fstat are properly implemented now):

Code: Select all

sbrk(0x28)
sbrk(0xFD8)
fstat(1, st)
write(1, ....)

If I write the application this way:

Code: Select all

#include <stdio.h>
#include <stdlib.h>
int main(int argc, char *argv[]) {
    printf ("Hello from user application!\n");
    return 0xBADC0DE;
}

This sequence is detected (sbrk and fstat are properly implemented now):

Code: Select all

fstat(1, st)
sbrk(0x418)
sbrk(0xBE8)
write(1, ....)

So, technically it's not newlib's printf have any dependency with sbrk in this particular case; nor does it invoke fopen().
Sorry if anyone is mislead.

Time of me to seriously read all the newlib document...

[TylerH]

Do you have a kernel heap? If you do, you could just have sbrk allocate from it. That's what I plan to do, if I ever get to the point of implementing a kernel heap.

[tomos]

Do you have a kernel heap? If you do, you could just have sbrk allocate from it. That's what I plan to do, if I ever get to the point of implementing a kernel heap.

That's exactly what I did, not really by plan either. I have just been working on a number of modules (not to completion, but enough to move on) and I made a kernel heap. I then decided to try the OS toolchain with newlib and I got it working today. Well, I haven't stress tested the library, but a few functions, namely printf are working.

So my sbrk call simply allocates a chunk from the kernel heap. This is all a temporary setup in my case, but it seams to be working. With one exception - I made a few calls to printf, then all the sudden it stopped calling my write() system call. I then tried calling fflush() and it printed. I've yet to figure it out (or to completely RTFM for that matter) but it might be of interest.

[pdurlej]

I then tried calling fflush() and it printed.

Default buffering modes depend on whether the descriptor in question refers to a TTY(4). You may want to double-check your isatty(3).