OSDev.org

Hello, OSDev.org.

Recently I started working on a kernel that would be able to run in both freestanding and hosted environments. I want to manually load ELF files and build host processes from them. How can I do it? Is there any way to create a "blank" process and control its memory and machine state under UNIX/POSIX?

Google told me that I can use ptrace on a "dummy" process started by fork/execv/etc. Is it a good way? Is there something better? What about forking the kernel and replacing the forked process from inside?

Ideally, I would like a way that is supported by many platforms, but I'm also interested in system-specific paths too.

I would recommend forking and replacing the child. Should be fairly easy to do actually. You can either cleanly discard the inherited data (open file descriptors and whatever else is passed from parent to child) or you can just ignore it and initialise your own data structures. Then you can either implement fork in your own operating system for standalone use or replace the process loader.

Something makes me think that there should be an exec that operates on memory rather than files, but obviously (for security reasons, I assume) most kernel developers decided that it would be better to avoid relying on processes correctly loading and executing an executable image (which, if it went wrong, could have drastic consequences) and rather try to enforce that the kernel does it itself.