Executable Programs

[Whatever5k]

Hi,
how do I manage that my OS can execute programs? Can I use existing file formats?

[K.J.]

You can use exsisting file formats. For docs on some of the existing formats check out:

http://www.nondot.org/sabre/os/articles
(this site doesn't seem to work right now but it did work last night, it's probably just being updated)

Hope that helps,
K.J.

[Whatever5k]

Allright, but how do I force the kernel to run this program? Do I have to take "call" in asm, or what else?

[K.J.]

Well, it seems that you and I are at the same place in our OS development. So, can anyone out there help us?

K.J.

[Bart Grantham]

I think the basic mechanism is like this:

You have a 'loader/linker' that you must build yourself (you can liberally steal sources for this).  It will load a program into memory and either 1) set up memory protection for a new process or 2) massage the symbol table so that the file is properly located.  It will also load shared libraries that the file may need.  That is terribly complex for getting things off the ground, so I suggest using only static linking for your programs for now. The sequence looks like this if you aren't using paged memory:

- Locate the executable on disk and load it into memory at either the location where the executable was statically compiled for, or somewhere where your kernel determines to load the program.
- If you need to, rewrite the symbol table so that all functions, data, etc. will resolve properly.
- jump to "_start"

Um... I haven't conquered this yet myself (still stuck on interrupt rewriting in PM) so I can't give too much more help, but that's the basic outline in my head as to my next step after interrupts.

Bart

[K.J.]

Well, today I was reading Tim Robinson's Memory Managment tutorials( http://www.themoebius.org.uk/tutes/ ) and I think that before we can run another program that memory managment has to working. Well after looking into the bootsector I'm using(bootf2.zip by John Fine) I discovered that it already enables memory paging. However, I'm gonna have to spend some time studding the code to understand it I think.

I'm guessing here, but I think that if we compiled programs as a flat binary like the kernel, there wouldn't have to be any messing around with header junk.

One thing that I do think will have to happen is that the kernel will have to assign the program a GDT at least if not also a TSS. As of now I haven't been able to find anything that really explains how the GDT and TSS work. Anyone know where I can find some info on them?

K.J.

[Bart Grantham]

Memory paging: I'm not so sure about that.  I mean, I think it is possible for you to be in PM without paging.  Also, seeing as how you can set up pre-emptive multitasking without paging, I can only assume that is not requisite.  I'm going to shoot for non-paged process creation first because I fear I'll be lost otherwise.  If you find documentation that shows it's impossible, please let me know so that I don't waste too much time.

I'm 100% sure that you need to set up a TSS for paged mem process creation.

Here's the doc that I've been reading.  It explains this stuff in painfully dry & explicit deatil.  The 386 Programmer's Reference: http://www.execpc.com/~geezer/os/386intel.zip

[K.J.]

Well, I think that I'll do memory paging 'cause I know that I'll want it later.

K.J.

[Norman Jonas]

If you want to create and load your own executables for your own operating system I would do it this way : ( at least with c/c++ and djgpp )

Write the program that should be executable using the headers of any functions that your kernel offers (e.g. printf ). Don't link it but create an object file that you know ( I use coff, it is easy and there is enough documentation to be found on it on the web ). For your operating system you have to create a symbol table with entries to names, that refer to a memory address ( like "_printf" is at 0xfffaaa ). You have to add any symbol that your own kernel wants to export ( This is why a loader is useful, because it can load the kernel as if it was a module itself and register all it's symbols ). Write a function like LoadModule ( char* pszFilename ) which loads the object file and "relocates" it. That means it replaces any address refference in the object file with the new "real" memory address ( a coff object file contains a special relocation list to tell which values in the code have to be changed ). Another function on the operating system side should be able to get any symbol address of the loaded module so that you can do something like GetAddress ( "_main" ). There is the entry point of the loaded module. Now call this address and you have run an external executable.

Note1: A symbol reffers to variables AND functions
Note2: The symbols "_printf" and "_main" are different when using C++ compilation.

This document is very general and may sound a little bit complex, but don't despare, once understood it's easy.
A good start to implement "dynamic linking" is to search the web for some source code on it ( There are several packages, like DXE, DLX, DLM and more, many of them for DJGPP ). But most of this packages, are bigger then neccessary to start with, and they want you to add some Macros to the loadable module, which is not neccessary. So I decided to use the above method.

Note3: To load and execute an MS-Dos executable or even Windows executable is somewhat different.

If someone thinks that more information concerning this topic would be useful, please tell me, maybe I could write a little tutorial including some example source code.

[K.J.]

Yes, please do write a tutorial, and I'll put it up on my site if that's okay.

I was just reading a post on alt.os.development and read that Windows uses software task switching. I've also heard that Windows uses paging. Hmm. Now I'm not so sure about the TSS stuff, I'll have to look at the 386 intel doc and see what I can find.

K.J.

[Tim]

I was just reading a post on alt.os.development and read that Windows uses software task switching. I've also heard that Windows uses paging. Hmm. Now I'm not so sure about the TSS stuff, I'll have to look at the 386 intel doc and see what I can find.

In real life nobody uses TSSes for task switching. At least one TSS is needed because the CPU gets the SS0:ESP0 pointers from there when it traps into ring 0 from ring 3 (or, in fact, from less-privileged code into more-privileged code). So at least that needs to be valid.

You will also need a TSS if:

• You want to selectively grant/deny user programs access to I/O ports (monolithic kernels will probably want to keep I/O access in kernel-mode drivers)
• You want to have 'special' handlers for things like double faults. If the kernel-mode stack breaks (e.g. if you have a stack overflow in kernel mode), and you use the same kernel stack for each exception handler, you'll get a triple fault when the CPU tries to handle the resulting stack/page fault. Having an extra TSS as your double fault handler will at least catch such errors and print a message instead of having the CPU triple fault (which will invoke a reboot on most computers).

Anyway, what was the point of this? Right... nobody uses TSS task switching because software switching is:

• faster (it's easier for the CPU to pipeline multiple PUSHes that it is to do one hardware task switch)
• simpler (you write the task switch code yourself)
• more efficient (you only swap what you need to -- most threads won't use the SSE2 registers, for example)
• more portable (I don't think any other architecture has hardware task switching)
• more reliable (a hardware task switch is all-or-nothing, making task problems hard to diagnose; if part of the software task switch goes wrong at least you'll know *where* it went wrong)

Paging (vs. segmentation) is also a good idea. Like hardware task switching, segmentation sounds good (each object gets its own little address space), but in reality, it comes down to portability again. There are no 386+ compilers out there that support a segmented architecture. The fabled OpenWatcom compiler (which Watcom have been promising for ages) is supposed to support full 48-bit far pointers, but for now, all 386 compilers expect to run under a flat 32-bit address space. The result of this is that, unless you want to be bound to assembly for your whole OS, paging is the way to go.

[K.J.]

Anyone happen to know how to access the memory set up in John Fine's bootf02.zip and how to set up more with a kernel loaded from that bootsector?

I'd be asking John Fine but the email address that I found for him doesn't exsist.

K.J.