IPC, Kernel Processes, Client/Server

[Trashey]

Hey all, this is going to be a messy post, so bare with me. (pretty please ;D)

I've come to a point in my OS where I have most "base" functionality working (process,dlls,drivers,threads,user envrionment etc) but this is where things can go really wrong.
Until now most of the things I wanted to write sat tight in my head and which I considered straightforward, but now I don't want to choose a poorly designed method spend tons of time on it and then regret, so I've decided to consult.

Processes can freely load and use DLLs which are mapped to their address space, kernel processes technically work, but have no support library at the moment and I started to fiddle around with how I would implement the client / server support.

What is a "kernel process" for you?
Is it perhaps a process that is loaded inside the kernel address space and uses the kernel address space?
Or is it a process that has its own address space like in a user address space (they both have the kernel mapped) but the address space is mapped as ring 0 unlike a user process?

How would I implement a kernel dll that can be called by a user program?
Consider the following example a "kernel" dll such as _xxx_32.dll (for example) can have both user mode code and kernel mode code, the dll is imported by a user process, considering that this DLL is only mapped to the user process, which then calls its interface through system calls which are implemented inside the DLL itself, so basically:
The dll registers the system calls at load time.
The dll exports only the user mode interface.
The user program calls the user mode interface functions (which are just a system call trampoline) which then traps into kernel mode and then back to a kernel mode function inside the dll, all is good.
However what if I want JUST the interface functions to be available in user mode and all the rest to be available in kernel mode? I would have to map only part of the dll privilege as user and the other part as kernel, who can assure me that the code does not share a page boundary?
(I don't want a user process to accidently jump into a kernel mode function of a user dll)

Regarding the Client Server, lets say I want my UI server to be a different kernel process, what I would do is register its functions as system calls, but is there a better way?
How would a process communicate with another process? I have read all that in the FAQ but I'm not sure what to choose and I was wondering how you guys implemented it, do you reserve inside each process a dedicated message area and perhaps an extra blocking thread that blocks until a message is available?

I know this is really messy, but I'm writing this message after an extensive coding period, if anybody can comment on how he implemented his IPC or Client/Server functionality or forward me to someplace it would be most helpful.

Thanks!

[Colonel Kernel]

You might want to explain why you've tried to implement things this way. I've never heard of mixed user-mode/kernel-mode DLLs before. In all the OSes I know of that support kernel modules, the kernel acts as a traffic cop between user processes and the functionality provided by the kernel modules (i.e. -- the kernel implements the "device" abstraction that processes in user space can manipulate via particular system calls).

I define a "kernel process" as one that runs in the kernel's address space. I suppose it could have an extended kernel-mode address space itself, but it doesn't make any sense IMO for it not to share at least the kernel's address space.

[Crazed123]

I would define a kernel mode process as one that is 1. Running in kernel mode processor protection, 2. Has the kernel mapped into its address space and 3. Accesses kernel code/data through a lower-level interface than usermode code that is created especially for kernel mode processes. For example, being able to make "system calls" that manipulated kernel data structures would be strictly for kernel processes.

This of course means that such processes better damn well by secure.

[JoeKayzA]

Hi!

About the 'kernel process':

I haven't yet heard about an OS that uses a full address space with ring0 privileges, and I suspect this has a reason: When would this approach really be neccessary? Probably when you are really running short on kernel address space. For all other scenarios however, you normally use a distinct address space when you want to isolate a bunch of code (like user mode code). When this code is then running at ring0 however, nothing prevents it from circumwenting the borders (disabling interrupts and switching to another address space itself) so you can't really isolate it that way. You would, however, cause additional overhead by needing to switch address spaces whenever this code is scheduled...

Therefore, IMO, ring0 code should go into the kernel space, which is then mapped into every address space. Everything that runs in a distinct address space should run at ring3, to ensure isolation.

A kernel space process is, in my definition, actually a kernel thread. A thread that doesn't have it's own address space, and thus no address space switching is needed when it is scheduled.

About mixed kernel/user mode dlls: Hmm, I don't see the reason why you want these to be mixed in one file...You are actually worrying only about seperating these two pieces of code in your post. So why don't you seperate them in two files? Or create an archive-style file that contains both, but load them in two parts?

cheers Joe

[Trashey]

Both user and kernel process has the kernel mapped into its address space. Kernel process runs in CPL0.

Consider the following functionality: a dynamically loaded kernel mode service.

The service should be accessible from user mode, but should have a kernel functionality.

How would you call a dynamic kernel module service from user mode?

If we choose the first method, ie, a kernel process uses the same address space as the kernel, then the following implementation is possible:

Two dlls will be supplied, one dll for user mode interface and the other will be the kernel mode service functionality.
When the user imports that service_user_mode_dll, the dll will load the kernel service dll, which will be mapped in the kernel space and register it's system calls.
The user will call the interface functions which are inside the service_user_mode_dll (and are just syscall stubs) which would then syscall trap into kernel and call the kernel mode functionality.

However, if we choose the second method, ie, a kernel process has its own address space (which ofcourse, shares the kernel address space), the previous implementation is no longer possible since syscall operate in the *calling process context*, and since the calling process is a user process with its own address space (and the kernel mapped) it only knows to call code inside the kernel address space. The actual service functionality exists inside a kernel mode process address space and not in the kernel address space and thus the system call cannot call code which is in a different address space.

The best example for this is the win32k.sys service in NT, at load time it registers the system calls, and holds a seperate user32.dll for the syscall stub user interfaces (which the user process imports)

I think I'll go for one, however, I don't like the idea that kernel processes will use the SAME address space as the kernel, this could run out of address space space

How would you however, implement IPC?

[Trashey]

Thanks joe, that's exactly what I'm doing.

[Legend]

Hi!

About the 'kernel process':

I haven't yet heard about an OS that uses a full address space with ring0 privileges, and I suspect this has a reason: When would this approach really be neccessary? Probably when you are really running short on kernel address space. For all other scenarios however, you normally use a distinct address space when you want to isolate a bunch of code (like user mode code). When this code is then running at ring0 however, nothing prevents it from circumwenting the borders (disabling interrupts and switching to another address space itself) so you can't really isolate it that way. You would, however, cause additional overhead by needing to switch address spaces whenever this code is scheduled...

In my latest design, the idea of a full address space running at ring 0 is one important aspect.
That is the address that a JIT would fill with program code, which does of course not need the ring 3 protection.
This means that a lot of programs will be in that single address space, too.
This might be a way to solve the problem of a full address space, because a JIT could be able to create another address space.

However, as most (and I, too) think that a design that only uses this might be too extreme for today, normal native code must be able to run in the system, too, and those would get a "classic" address space.

This however creates the need for the scheduler to take advantage of different amounts of processes in namespaces to reduce the amount of cr3 switches, up to the point where none are necessary.

[JoeKayzA]

This however creates the need for the scheduler to take advantage of different amounts of processes in namespaces to reduce the amount of cr3 switches, up to the point where none are necessary.

Moin! (=> Hi!)

I think that depends on whether you schedule processes, or you schedule threads. When you schedule only threads, you could check if the thread's address space is different from the active one, and if not, no adjusting of cr3 is needed. At the next step, you'd need a scheduling optimizer, that puts the threads in an order which will cause the least possible cr3-switches (and, of course, also considers other things). This should do the job then.

Another thing I thought about recently:

When you are really mixing up these concepts like this, you should also consider that there will be 'safe' components which are called quite often by 'classic' components, drivers, codecs and the like. Then you should put them into the kernel-portion of the address space, which is mapped into all other (ring 3) address spaces, making them mostly like 'kernel modules' in the classic sense. This would avoid further unneccessary context switches then. The counter-point here is that the kernel space is probably very limited (1GB?). So you have to choose only those components with the highest usage. All those which don't fit in there will have to be put into the ring0-only address space of its own - just another idea of optimization.

cheers Joe

[Legend]

Yes, in some way it is only halfway new, but it should not be forgotten while implemeting!

Placing some components into the kernel space might be an optimization, true. Especially if the JIT opens two ring 0 address spaces. As you pointed out, placing a lot of components in the kernel address space (and then reserving more space) creates a greater need for several address spaces!
However, only if you really get into the region of several gigabytes of data.