How Kernel, Drivers and Processes work together

[sebihepp]

Dear Forum,

again I was suddenly understanding some part of OS theory much better as I walked in my appartment. This time it is about the communication between the kernel, drivers and processes.

For a long time I thought I could implement drivers like objects of classes. An process opening a file would call fopen(), which would then call something like FileSystem[0]->ReadFile(Buffer). Just like programming DirectX 9, getting a D3DDevice and then call methods of it to display graphics.

Man was I wrong.

The problem is, that user space processes can't just access methods and data of drivers. The drivers are either seperate processes (Microkernel) or in kernelspace (Monolith). That makes IPC (inter process communication) respectively systemcalls to the only way to interact with drivers. That way handles make really much sense now as a way to index different ressources.

For my OS I would like to have drivers as seperate processes, but running in ring 0, so they have access to kernel ressources.

Now I have just one question: How to handle multiple devices with the same driver? (like multiple AHCI devices)
Do you instantiate one process of the driver for every device? Or do you specify, that the driver has to have a way of handling multiple devices?

Best regards
Sebihepp

[nullplan]

Funny you should mention object oriented programming, because I have always seen the file descriptor abstraction as using basically virtual methods. Yes, you call the read() system call, but what will it do? On a disk file, it will read disk file contents. On a socket, it will read from the network. On a pipe, it will block until you get bored. On a GPIO instance, it will return an error, because you are only supposed to use the ioctl() interface there.

Just because the object orientation is hidden behind the syscall veil doesn't mean it isn't there.

Now I have just one question: How to handle multiple devices with the same driver? (like multiple AHCI devices)

Typically you just tell the instances apart. A driver should not have global/static variables, but rather should have everything that is specific to the instance in some structure. For the specific case of user visible devices, the driver should tell the kernel that it has some device, e.g. a disk, and accessing it requires accessing the driver instance.

[sebihepp]

Typically you just tell the instances apart. A driver should not have global/static variables, but rather should have everything that is specific to the instance in some structure. For the specific case of user visible devices, the driver should tell the kernel that it has some device, e.g. a disk, and accessing it requires accessing the driver instance.

It is still not completely clear to me. So a driver is the implementation of a class with an additional extern "C" CreateInstance() function, that has ports and mmio addresses as parameters and returning a pointer to a newly created Instance of the class. Then the syscalls can use the object and call its methods. But then a driver is more like a dll than a process. And the kernel would be more a collection of classes with methods, that can be called by programs. That would also require that all drivers always need to be mapped into every process. And the line between kernel and driver is fading away.

The other way would be more like a Server/Client approach with message passing between processes. After the driver is loaded its main() function will be executed. In that function it uses syscalls to read its message queue and react to it. Programs then would use syscalls to pass a message to the driver and the driver answers with a message once the requested work has been done. Shared Memory could be used to exchange larger arguments.

Hmmm, That doesn't sound right. I need time to think about it and come up with alternatives.

[nullplan]

The idea is that the driver sends all the data that identifies an instance to the kernel, to be sent back on requests. Let's take disks/volumes for example. A disk knows how big a sector is, how to read and write one, and how many there are. So you'd model it in C like this:

Code: Select all

struct disk;
struct disk_ops {
  int (*read_sector)(struct disk *, void *, off_t);
  int (*write_sector)(struct disk *, const void *, off_t);
};
struct disk {
  size_t sector_size;
  off_t sector_count;
  const struct disk_ops *ops;
  struct list list; /* for keeping track of the disk. */
  char *name; /* filled in by register_disk */
};

That is as general as it gets. The kernel core then has some function

Code: Select all

int register_disk(struct disk *);

And finally the AHCI driver then has its own specialization of this, like

Code: Select all

struct ahci_disk {
  phys_addr_t membase;
  /* whatever other data needed */
  struct disk disk;
};

And then when the driver is instantiated, it fills out the disk structure and registers it with the core. And the functions turn the disk into an ahci_disk with the container_of macro.

That way then the rest of the kernel knows there is a disk to use, and can for example attempt to discover partitions on it. A partition abstraction can be built on top of the disk abstraction and it only has to know about partition tables, not access methods.

In C++ you'd probably be less crude about this and just use class hierarchies and virtual/abstract methods.

In microkernels you do much the same, only the driver registration must be a system call. I mean, by the time the PCI layer instantiates the driver, it cannot know what the driver will turn out to be, right? It might be a disk driver, or a serial line driver, or a graphics card driver, and the PCI layer shouldn't assume. It could also be that initialization fails, and then the driver won't be anything. Of course, in microkernels you wouldn't pass function pointers; rather the kernel would have to serialize the requests and send them to the driver on a socket or something.

And the kernel would be more a collection of classes with methods, that can be called by programs.

That is part of what a kernel is. A kernel is a library for doing privileged stuff in a safe way.

That would also require that all drivers always need to be mapped into every process.

In monolith land (which I'm a citizen of) that is the case anyway. And in microkernel land, the function interface is replaced with an RPC interface and the driver can be a different process.

After the driver is loaded its main() function will be executed. In that function it uses syscalls to read its message queue and react to it. Programs then would use syscalls to pass a message to the driver and the driver answers with a message once the requested work has been done

Correct but incomplete. Crucially, when the kernel spawns the driver, it doesn't know what kind of driver this thing is going to be, or if it is going to be any driver at all. So the driver has to register with the kernel to tell it what it is. In this case for example that it is a disk and how big it is and how big its sectors are. That could also be the place where the communication channel is established. So if I was making a microkernel, I would adapt the above design to have

Code: Select all

int sys_register_disk(size_t sector_size, off_t sector_count);

as a system call. Drivers can call it and it returns a file descriptor to a socket that the kernel uses for requests.

There is always going to be a kernel core part that handles the driver instances. The AHCI driver only knows how to talk to AHCI disks, but there is a partition driver that creates the partitions, there is a file system driver that implements the FS. And these need to take disks as inputs. For example by passing the channel for the disk to them.

And applications are going to want to read and write files, so the entire thing is far removed from the actual disk driver. The application wants to do something with a file, then the VFS layer has to figure out what FS this is on, the FS layer has to figure out where to store the data on the volume. If this is on a partition, it has to add the starting offset, and only then can the disk driver get involved.

[rdos]

There are typically many levels to this.

For handle-based IO, user space will typically pass a pathname, and possibly some other data to open a handle. The handle is then used to read or write. User space doesn't need to know (and shouldn't care) if it has opened a socket, a file on an AHCI disc, or standard input/output.

In kernel space, there will be a handle table that remembers how the handle can be accessed. This can be viewed as classes and objects with virtual methods, but it doesn't need to be implemented that way. The first level, therefore, defines which device is responsible for a specific handle. It can be a pointer to a device structure and potentially a cluster number or something so the file can be remembered. Or it can point to a socket or standard-IO. The device "class" might contain methods like read, write, duplicate and close. There is also a need to keep a reference list for each handle since many processes/threads can have the same handle open. Also, the handle table must be per process, and it should be cleaned up on process termination.

I don't think monolithic vs microkernel enters the picture until the next level. I have a mixed monolithic and microkernel VFS, and the device class methods will know the difference. Read using the server process interface will issue requests with IPC, while the legacy-interface will call kernel functions.

My device discovery scheme is the reverse compared to the one described above. The AHCI driver will enumerate PCI devices and then install disc drivers for those with an AHCI device class. The IDE driver will probe standard IO addresses, but will also ask for PCI IDE devices.

Handling partitions is another level. There is a need to create a uniform pathname tree, which requires the linking of partitions to the tree. I use the drive letter method, which might be a bit easier since each partition just needs to allocate a drive letter. When a pathname is parsed, this might traverse various partitions, particularly if links are supported.

[sebihepp]

Crucially, when the kernel spawns the driver, it doesn't know what kind of driver this thing is going to be, or if it is going to be any driver at all. So the driver has to register with the kernel to tell it what it is.

Thanks. That was one thing I was wrong about. I thought the Device enumerator function loops through devices and then loads the specific driver per device. I was thinking too much in DirectX 3D ways. There you enumerate Adapters with a D3D9 Object and then call D3D9->CreateDevice(Device) to get an D3DDevice9 Object, which represents an Adapter (GraphicsCard) and has methods like Clear (for clearing the screen), etc. I also thought it must be this way, because I wouldn't want every driver to search for devices itself - it can't know which resources are available on the computer.

But it makes much more sense, that you call init of a driver and the driver registers his type at the OS. The OS then calls Driver->Instantiate with infos about the device, like IOPorts Base and MemoryMappedIO Base for example.

I my mind I want the OS not to provide the functions, but only to manage resources. MemoryManager, Scheduler and IPC are the only things my kernel should do. If a program wants to open a file, it should call the respective driver. But it seems I can't do it that way.