System Calls and Returning Quickly From Interrupts

[Stevo14]

My understanding is that modern operating systems stress returning from interrupts as quickly as possible. My question is how is this done feasibly with certain system calls?

For example, I'm currently trying to write a SYS_EXEC system call. Obviously such an operation (starting another process) is not quick. In this particular example there also needs to be disk IO access in order to load the program. The disk access doesn't work inside the syscall handler though because the disk IO functions rely on other interrupts (which don't seem to come while inside the interrupt handler). Note: All disk access is synchronous at this point because I'm still single-tasking.

Can anyone give ideas for a possible solution?

[thooot]

OS's try to return from hardware interrupts as quickly as possible but for software interrupts (a.k.a systems calls) its okay to take a long time. For instance you may have an OpenFile system call and if you pass it a network path you need to establish a connection, send a request, process the response and finally return to the caller. All of which could easily take multiple seconds depending on the latency. This is of course assuming the system call is synchronous. If you really want the system calls to complete quickly you can make them asynchronous (queue up a workitem to a thread pool to execute the call and return to the user immediately). However asynchronous calls are more complex for applications to use. They do potentially have the benefit of being easy to cancel (depending on implementation) if they are taking too long.

[proxy]

the traditional solution is basically to do the minimum amount of work to know the return value/success and do the rest in a "bottom half" handler which is effectively a kernel mode thread to be run at a later point in time. Not all interrupts/system calls lend themselves to this strategy, but it can work very well for those that do.

[Stevo14]

OS's try to return from hardware interrupts as quickly as possible but for software interrupts (a.k.a systems calls) its okay to take a long time. For instance you may have an OpenFile system call and if you pass it a network path you need to establish a connection, send a request, process the response and finally return to the caller. All of which could easily take multiple seconds depending on the latency.

So it is okay to take a long time in software interrupts. Good. That makes things a lot easier. Would it then be okay to enable interrupts while inside the system call (which I need in order to do disk access etc.)? Actually, the whole new process would technically run inside the interrupt handler so it could take an indefinite amount of time to return and would need all hardware interrupts enabled for timing, keyboard input etc. I can't think of any immediate problems that would arise, being that this is all single tasking. Good idea or no?

The "bottom half" handler sounds like a good idea but currently my kernel has no concept of either threads or processes (thats actually what I'm working on now) so I would have to implement it in a different way.

[Brendan]

Hi,

For example, I'm currently trying to write a SYS_EXEC system call. Obviously such an operation (starting another process) is not quick. In this particular example there also needs to be disk IO access in order to load the program. The disk access doesn't work inside the syscall handler though because the disk IO functions rely on other interrupts (which don't seem to come while inside the interrupt handler). Note: All disk access is synchronous at this point because I'm still single-tasking.

Can anyone give ideas for a possible solution?

For this specific example, (depending on kernel design) it's possible to start a bare process (without doing any file I/O) and return, and let the new process load it's own executable.

In this case you'd create a new address space, create a first thread for the new process, send some information to the new thread (executable's file name, command line arguments, environment strings, etc) and then return. The new thread does the rest of the work - it starts executing kernel code that determines what sort of executable file it is, and does whatever is necessary to execute that file - for e.g. load the executable's header, load the executable's sections into the address space where the header says each section should go (or alternatively use memory mapped files), load or map any shared libraries (if necessary), do any dynamic linking (if necessary), then find the entry point and "return" from the (CPL=0) kernel's process loader to the (CPL=3) executable file.

Of course the kernel's process loader could support more than this. For example, if the executable file is a shell script then load a shell to interpret the script, if the executable file is java byte-code then load a java byte-code interpretter, if the executable is a SPARC executable then load a SPARC emulator, etc. That way a programmer wouldn't need to care if "FOO" is a native executable or something else when they call SYS_EXEC (which means if FOO is a script they can replace it with a native executable without modifying any code that starts FOO).

If you do it like this then the SYS_EXEC system call can be fast, but it won't be able to return many errors - if the bare process can be created the system call would return "OK", regardless of whether or not the executable file can be loaded and started. You'd need to use something else to determine if the new process actually did start correctly (for e.g. the kernel's process loader could send a status message back). Of course you need something like this anyway, so that the parent process can get status back when the child process calls "exit()" after it's been running.


Cheers,

Brendan

[Stevo14]

For this specific example, (depending on kernel design) it's possible to start a bare process (without doing any file I/O) and return, and let the new process load it's own executable.

... [the rest of the post] ...

I absolutely love this Idea, but unfortunately it won't work for me because my kernel, as of this post, has no concept of a process or thread. Simple program loading is my first goal and then I will get on to multitasking and threads.

I have tried simply allowing interrupts inside the syscall handler and it seems to work without problems. Other, more creative, solutions are still welcome though.

[Brendan]

Hi,

My advice is to start small...

Begin with code to spawn a "kernel thread" (a thread that runs at CPL=0 in kernel space). Then write a routine to switch to a specific thread (e.g. "jumpToThread(THREAD_ID thread)" and test it (try to break it or crash it in as many ways as you can). For example, try spawning 2 kernel threads that constantly jump to each other, then spawn an infinite number of kernel threads in a loop and see what happens.

Next, write your scheduler. The scheduler would be something like a "switchThreads(void)" function which chooses the best thread to run and then calls the "jumpToThread(THREAD_ID thread)" that you've already written (and tested). Don't forget to test it (e.g. try spawning 100 kernel threads that all constantly call "switchThreads()" and see if you can break or crash your code).

Next, add some preemption. This is simple enough - it's a timer IRQ handler that calls your "switchThreads(void)" function. Now test it thoroughly!

Next, write some code to allocate and free address spaces - for e.g. "ADD_SPACE_ID allocAddressSpace(void)" and "freeAddressSpace(ADD_SPACE_ID addressSpace)". Test these too. For e.g. have 50 kernel threads that continually allocate and free address spaces, and see if that causes any problems.

Now, spawn a new kernel thread that allocates an address space, switches to the allocated space, then puts the instruction "jmp $" at address 0x00000000, then does "jmp 0x00000000". If that works remove the "jmp 0x00000000" and replace it with some code that pushes EFLAGS, CS and EIP onto the stack and does IRETD. If that works, change the value you push on the stack for CS so that it's a CPL=3 code segment instead of the kernel's code segment. If that works you've got a user-level thread.

Next, add some code that builds some sort of structure that describes a process (which includes a "totalThreads" variable). Then add a kernel API with a "terminateThisThread()" function that decreases the "totalThreads" variable in the process structure. If the "totalThreads" variable in the process structure becomes zero, then the process has no more threads and needs to be terminated too (free the address space, etc). Test this by replacing the "jmp $" code that your process is using with code that calls your "terminateThisThread()" kernel API function. Now you should be able to continually spawn processes that all terminate themselves.

Lastly, instead of copying some dummy code into the process/thread's address space, load it from disk. I'd start with something insanely simple (e.g. binary executable format with no headers or anything). When that works you can replace it with an ELF loader or something.

Note: The idea I'm suggesting is to work in small steps and to thoroughly test each step as you go (rather than writing all of it before testing any of it, then spending ages finding all the bugs before you see any of it actually work). You don't need to take any of the steps literally - the steps I listed are just an example (feel free to make up your own steps that suit your OS design).


Cheers,

Brendan

[Stevo14]

Brendan,
Thanks for writing that mini, multitasking walk-through. Ill keep this thread handy for when I start working towards multitasking (which might actually be soon).

As of right now, I have a working implementation without threads where SYS_EXEC loads a flat binary (from the file system) to 0x10000000 and jumps to it. The flat binary prints some messages (using some message-printing system calls) and calls SYS_EXIT, which causes control to return to the parent right after where it started the process. I guess the next step is either ELF parsing or threading... hmm... which to choose...