L4, Posix and Drivers implementations

[GianlucaM]

Hello everybody,
I'm Gianluca Magnini, a student in Computer Science in Siena (Italy). This is my first post in the forum!
Actually, I'm studying about various L4 Microkernel implementations (Pistachio, Codezero, OKL4), but I have some doubts about how to implement functionalities user-oriented.

In fact, I would like to understand how to implement a Posix layer and use it to implement Device drivers as user-servers. I've looked for documentation in OKL4, Pistachio and Codezero manuals, and furthermore I've tried to undestand the code of the three Microkernels, but it seems that all of those Microkernel uses some frameworks (such as Genode, Iguana, or similar) to add some abstraction layer between Microkernel and user-threads.

I would like to implement the Posix layer directly using the Microkernel functionalities, and the Drivers directly using Posix layer abstraction. I tought about the creation of a Posix server that is launched at the beginning, and that contains data structures and functions capable of serving "posix requests", such as pthread_create. Each Posix is so an IPC to that server. I think Codezero has a similar implementation, in which the Posix server is called VFS. My problem is that I don't understand how to implement read/write functions from User thread to Device driver. I explain it better: suppose I've an user thread that would like to read an ethernet packet, using "read" function. The user will "open" the ethernet device and will read from it.

User code:

Code: Select all

[...]
uint8_t buf[4096];

eth = open("eth", "r");
read(eth, buf, size); 
[...]

This translate in two IPCs (one for Open and one for Read) to the Posix server that will create a sort of file descriptor in the first case (Open) and will read the data in the second case, transferring the data from the driver to the user (Read). Anyway, I cannot figure how to pass the data between Driver (and Driver address space) and User (and User address space) without sharing sensible data! I mean: the user buffer is in the User address space, and the Driver buffer is in the Driver address space. Should be the Posix pager to copy a buffer to another? And how it is supposed to do it without making a mapping between the two address spaces?
I cannot use Iguana or other services since my need is to develop these services in the most direct way...

Something like the picture below (Sigma0 excluded, which function is only to launch Posix pager and manage memory requests from it).
+------------------------+
Drivers | User Thread
+---------+------------- +
Posix pager
+------------------------+
uKernel
+------------------------+


Thanks for your help
Gianluca

[OSwhatever]

The driver is a user process just like any other user process. A posix layer is most likely to be a layer in user space as a shared library, then the posix layer calls various services in user space and/or direct kernel calls depending on the posix function.

Yes, in order to transfer data using IPC, the kernel is involved in the actual copying of the data and the driver/server is as secure you make. If you don't want share sensitive data, you simply don't share it. How the kernel is copying/transferring the data may vary a lot. The kernel can for example temporary store the IPC data in the kernel for small sizes. It can copy from one address space to another by mapping a part of the receiver's address space in the kernel. There are several methods here.

One thing that is obvious here I think is that the posix "everything is a file" isn't really suitable for a microkernel. A microkernel is more client/server based using IPC. "everything is a file" was suitable for monolithic kernel as a method for communicating between user processes and user/kernel. With a defined IPC interface that file abstraction can make thinks more complicated that it really is.

[GianlucaM]

Thank you for your reply.
There is a thing that I do not understand (or, better, there are many things, but especially one ): how can I implement a Posix layer using only IPC? How can I maintain structure to manage everything, if everything is distributed?

I mean..posix library should be compiled with the user application, right? So each management structure should be maintained and synchronized by each thread agains the others?

Thanks
Gianluca

[OSwhatever]

Thank you for your reply.
There is a thing that I do not understand (or, better, there are many things, but especially one ): how can I implement a Posix layer using only IPC? How can I maintain structure to manage everything, if everything is distributed?

I mean..posix library should be compiled with the user application, right? So each management structure should be maintained and synchronized by each thread agains the others?

Thanks
Gianluca

I don't think management structures differs too much from a monolithic kernel, some calls might want to store data in user space but it's not really much what I've encountered. In practice your posix call (might it be read, sbrk and so on) is just replaced from a kernel call to an RPC call using the IPC.

[gerryg400]

One thing that is obvious here I think is that the posix "everything is a file" isn't really suitable for a microkernel.

Perhaps but what about "everything is an fd-that-describes-a-connection-to-a-user-mode-service-that-looks-like-a-file". This is basically the idea that QNX use to make their Posix OS on a microkernel.

Note that what I describe here is not, I don't think, exactly the same as the QNX implementation.

OPEN might look like this:

Code: Select all

1. client calls eth_fd = open("eth", "r"); which does 3 things
      a) sends a message to the VFS name server that returns the mailbox ID of the "eth" driver.
      b) asks kernel to create an FD that maps to that mailbox ID. The FD is in the kernel
      c) sends an "open" message to the "eth" mailbox via that FD

2. eth server receives the msg, checks permissions and if all is okay creates an file-control-block to represent the "eth" file.

3. eth server makes a kernel call that acknowledges the FD can be opened from the client to "eth". This step is important because it ensures that the kernel knows that both parties agree to communicate. But the kernel will be unaware of the content of the communication. At this point the "eth" server supplies a cookie to the kernel that identified the FD from the server side. The cookie is a pointer to the file-control-block.

4. eth sends OK status back to client.

READ might look like this:

Code: Select all

1. client calls read(eth_fd, buf, size); which does 2 things
      a) sends a read message via the eth_fd. The kernel converts that to a mailbox ID and supplies the cookie to the server.
      b) the server doesn't see the FD but recognises that the cookie is a file-control-block.

2. Eth server sends the ethernet data back to the client. It doesn't need to do any security checks. This can be done with shared mem, copy etc.

CLOSE might look like this:

Code: Select all

1. Client makes kernel call that closes FD

2. Kernel sends a message to eth server that FD has been closed and then destroys the FD.

3. Server destroys file-control-block and the entire connection is closed.

[OSwhatever]

Perhaps but what about "everything is an fd-that-describes-a-connection-to-a-user-mode-service-that-looks-like-a-file". This is basically the idea that QNX use to make their Posix OS on a microkernel.

Note that what I describe here is not, I don't think, exactly the same as the QNX implementation.

OPEN might look like this:

Code: Select all

1. client calls eth_fd = open("eth", "r"); which does 3 things
      a) sends a message to the VFS name server that returns the mailbox ID of the "eth" driver.
      b) asks kernel to create an FD that maps to that mailbox ID. The FD is in the kernel
      c) sends an "open" message to the "eth" mailbox via that FD

2. eth server receives the msg, checks permissions and if all is okay creates an file-control-block to represent the "eth" file.

3. eth server makes a kernel call that acknowledges the FD can be opened from the client to "eth". This step is important because it ensures that the kernel knows that both parties agree to communicate. But the kernel will be unaware of the content of the communication. At this point the "eth" server supplies a cookie to the kernel that identified the FD from the server side. The cookie is a pointer to the file-control-block.

4. eth sends OK status back to client.

READ might look like this:

Code: Select all

1. client calls read(eth_fd, buf, size); which does 2 things
      a) sends a read message via the eth_fd. The kernel converts that to a mailbox ID and supplies the cookie to the server.
      b) the server doesn't see the FD but recognises that the cookie is a file-control-block.

2. Eth server sends the ethernet data back to the client. It doesn't need to do any security checks. This can be done with shared mem, copy etc.

CLOSE might look like this:

Code: Select all

1. Client makes kernel call that closes FD

2. Kernel sends a message to eth server that FD has been closed and then destroys the FD.

3. Server destroys file-control-block and the entire connection is closed.

I see, so in practice they create a new channel which gets a unique id for every open file. It's quite brilliant really, I withdraw my previous statement then. Then you need the channel/mailbox/port abstraction so L4 would not be able to do this?

[gerryg400]

It's quite good. Because the kernel manages the fds it can inform the client or server immediately if the other one dies. Security checks are never performed by the kernel so messages can be passed quickly and because the fds themselves are kernel-managed, security checks only need to be done by the server when the FD is created. If the server hasn't acknowledged the FD, the client can still use that FD but the server will know to ignore the message.

There are other benefits too. For example Posix ideas like fork(), where all the fds need to be dup'ed can fairly easily be implemented.

[OSwhatever]

This method pretty much require channel abstraction and that they are bi-directional. Now if you don't have bi-directional channels you would need some local file handle that translates into either read or write channel id. This can be done however, but this serves as a good example when local management structures are needed in user space which GianlucaM was originally asking for.

[GianlucaM]

I would like to thank you for your answers.
Thanks to your suggestions now I have a more clearer vision of what to do to implement a sort of Posix-Layer inside the Microkernel.
It will manage the communication between user threads and drivers or other threads using Posix API, that will be based on IPC and data transfer. I think it should be fine (of course I've to think a little bit more about how it can be implemented).

Thank you very much, I will update you about the development

GianlucaM