Porting the kernel as a user mode program

[Thomas]

Hi Andrew,

Very Good Thinking!. Also it is far more easy to test in user mode. You already might have heard of user mode linux - UML. http://usermodelinux.org/ . There is a book on amazon about it which may prove helpful : http://www.amazon.com/User-Mode-Linux-J ... mode+linux .


--Thomas

[Thomas]

Hi Andrew,

Just forgot about tunix :- http://homepages.thm.de/~hg53/tunix/ . It is an educational unix system that runs as a unix process. It might also prove helpful.

--Thomas

[AndrewAPrice]

I would rather have my OS that can be compiled to run on either baremetal or as a usermode program, than having two separate programs (a baremetal kernel, and a user mode VM).

The complexity will be at least comparable with the VM variant. But if you like it - why not?

Because I believe I'm going to solve many problems and implement a lot of complex logic, that it's better to simply wrap elements as in:

Code: Select all

bool vfs_readfile(FILE *f, uint64 start, uint64 length, void *buffer) {
#ifdef BAREMETAL
     // read from the disk
#else
    // call the host OS
#endif
}

For example, the scheduler only compiles when running native code, otherwise I can use the host's threads. My window manager can use the host's windows.

[Thomas]

I would rather have my OS that can be compiled to run on either baremetal or as a usermode program, than having two separate programs (a baremetal kernel, and a user mode VM).

The complexity will be at least comparable with the VM variant. But if you like it - why not?

Because I believe I'm going to solve many problems and implement a lot of complex logic, that it's better to simply wrap elements as in:

Code: Select all

bool vfs_readfile(FILE *f, uint64 start, uint64 length, void *buffer) {
#ifdef BAREMETAL
     // read from the disk
#else
    // call the host OS
#endif
}

For example, the scheduler only compiles when running native code, otherwise I can use the host's threads. My window manager can use the host's windows.

I think its not good design to have lot of conditional compilation.

Ideally you would have a disk abstracted as a file so you would have separate driver to read from the file instead of an actual disk. Similarly a processor might be abstracted as a process so all processor specific code goes to a separate file that deal with a 'process' processor.

vfs is supposed to be a high level layer devoid of platform dependent code.

--Thomas

[Wajideu]

I think its not good design to have lot of conditional compilation.

I agree. When I'm working on large C projects that have functionality with multiple drivers, I tend to use vector tables rather than pre-processor directives.


I like this idea a lot and I'm curious about the practicality of it. Is it going to be a full virtualization layer, or is this going to work by using relocatable code files?

[b.zaar]

I have been thinking about something like this lately. Would it be considered a good idea to design a simple VM library, similar to a standard C library, that an OS driver calls?

The idea I'm thinking is your kernel would call something like ATA_READ to a real kernel driver, then the driver you plug into the kernel would call VM_ATA_READ in the library without directly reading & writing to I/O ports. Each kernel would still need custom drivers but these drivers would all use VM_xxx system calls to a user level VM library. The VM library would provide common known calls for different hardware. This might work like a kernel calls VBE_SET_MODE, the driver would then call VM_VBE_SET_MODE. Other calls might be VM_PCI_xxx, VM_ISA_xxx, VM_NET_xxx.

With a design like this an OS developer would still learn real driver design by writing a driver that uses real things a hardware device supports without having to learn/test/debug the real world I/O control. The VM library could be ported to any OS and eventually real hardware drivers could be written to replace the virtual drivers when the driver interface is already designed and stable.

[Wajideu]

I don't know... That seems like overkill to me. The more abstract and modular the better imho.

[b.zaar]

It would mean the kernel can be compiled without modification, only the drivers would need to be changed. The vm_lib would run like a system debugger so you could have breakpoints and memory dumps. The kernel would run at the user level and the debugger would handle any traps caused by priveleged instructions, like a simple vm386 monitor. The main idea is so the kernel doesn't need to be aware it's not running natively on the hardware, only the drivers know they are not accessing the hardware directly.

Think of the drivers using a BIOS/UEFI type interface instead of directly accessing the hardware. The kernel requests to read a sector, the driver requests a BIOS/UEFI read sector service and the vm_lib reads a device. Either a disk image or a real device. The vm_lib would load the kernel from a regular file using multiboot or UEFI with no real boot sectors or partitions used. The OS could use a disk image, zip file or a plain directory as it's disk, it never really touchs the real hardware disk tables. The vm_lib could use a portable library like SDL to allow fulll screen mode and changing resolutions.

Does anyone know or use an Open Source debugger that could be forked/mangled for this kind of thing?

I think it's something that would be shared and it might increase the running speed and dev/testing cycles.

[Wajideu]

Does anyone know or use an Open Source debugger that could be forked/mangled for this kind of thing?

There's the GNU debugger, gde.

[b.zaar]

I think we might be better of starting with a windows version of this thing (if it's going to happen).

I've found this https://code.google.com/p/opendbg/ and it seems to be a good start. It can already handle user and kernel mode apps.

[b.zaar]

I've started working on this. The idea initially was to work with a debugger and let the OS run almost natively. I found a better solution and it seems to be progressing nicely, just slowly.

I'm using DOSBox as the base system and I'm intending to write a simple firmware that any OS can call. The beauty is DOSBox already runs on many OS's so that all you need to do is patch the 0.74 source and compile it to suit your needs. It also includes an internal debugger.

The system I've hooked is the internal command BOOT so you can run your own OS with the command

Code: Select all

dosbox -c "MOUNT /images/hdd500mb.img" -c "BOOT --firmware fd01.img fd02.img -l c"

The plan is to be able to create multiple firmwares that you compile into your standard DOSBox source code. You can then boot using BOOT --coreboot or BOOT --bios32 or BOOT --myownfirmware and your OS will be loaded with the installed firmware. The firmware exists outside of the DOSBox memory leaving more memory inside DOSBox for your OS.

Your default (or included -conf) configuration file can still run any auto mounts using the autoexec section. Your OS will be able to access the real filesystem through the firmware, just not using read or write sector calls.

At the moment I've completed a service callback (an internal system call to run a C++ function), entering 32bit protected mode from the C++ firmware, and I'm working on the multiboot loader.

I'd like to know if someone else is interested in using this mod?
Does anyone have any input about the firmware to use/create?
And the format of the configuration file for loading kernels and modules?

For proof DOSBox is a good start