Device Drivers' Interface considerations

[XCHG]

I am going to design a device drivers' interface for common devices. These are the steps involved in the communication between my kernel and my device drivers. I would appreciate it if you could give me hints, suggestions and etc:

• Device Drivers are flat binary files
  [1] The Device Driver is loaded from the disk.
  [2] The kernel calls the device driver's code at the offset of 0x00000000.
  [3] The device driver will have a function at offset 0x00000000 that will return two values. The return value will be the size of the structure that this driver needs to output its functionalities and one other return value (through a pointer parameter) will be the offset of the initialization procedure/function.
  [4] The kernel will take the return value in Step 3 and allocate a physical memory.
  [5] The kernel will then call the offset of the initialization procedure in the device driver, returned at Step 3 and passes the allocated memory to that procedure.
  [6] The device driver will fill in that structure with information such as the number of functions that are available in the device driver and their offsets, whether this device driver handles an Interrupt or IRQ (if yes, the offset of the IRQ/Interrupt handler) and etc...
  [7] The kernel will then add this device driver to a queue. This will allow various device drivers to be hooked to a single device, running simultaneously.

There are some problems with this approach of course. For example, the kernel might call the offset of 0x00000000 to a file which is believed to be a device driver. If this is not an actual device driver, an exception will happen. The kernel will then have to handle that exception and etc.

I would appreciate your comments on this scheme.

[Craze Frog]

Device Drivers are flat binary files

Which means that you're not planning to relocate them, right? How are you planning to load multiple drivers?

[XCHG]

Relocation as swapping them on and off the memory to disk storage and back? I am not sure what you mean by relocation in this case.

About multiple drivers, my kernel will be charge of managing a queue of drivers in line to service various interrupts and IRQs. So once the driver is detected, the kernel will place it in the queue. When an IRQ is fired, the kernel will check for all device drivers that are supposed to handle that IRQ. Let's say it finds two of them. It will execute them in the FIFO manner. The second one should be noticed of the changes that the previously-loaded driver has made to the memory/IO/etc or unless it could possible duplicate the same thing the first driver has done. What do you think?

[Brendan]

Hi,

[1] The Device Driver is loaded from the disk.

How do you load the disk driver from disk?

[2] The kernel calls the device driver's code at the offset of 0x00000000.
[3] The device driver will have a function at offset 0x00000000 that will return two values. The return value will be the size of the structure that this driver needs to output its functionalities and one other return value (through a pointer parameter) will be the offset of the initialization procedure/function.
[4] The kernel will take the return value in Step 3 and allocate a physical memory.
[5] The kernel will then call the offset of the initialization procedure in the device driver, returned at Step 3 and passes the allocated memory to that procedure.
[6] The device driver will fill in that structure with information such as the number of functions that are available in the device driver and their offsets, whether this device driver handles an Interrupt or IRQ (if yes, the offset of the IRQ/Interrupt handler) and etc...

Why can't the device driver's initialization function allocate some memory (if necessary) and return a pointer to a structure of information (instead of having 2 initialization functions)? For simple devices, the structure of information might already be in the device driver's ".data" section (no memory allocation needed)...

What will be the format for this structure of information? Will it be the same format (same fields, etc) for all device drivers (e.g. keyboard driver, hard disk, SCSI controller and video card)? It'd be extremely difficult to discuss the device driver interface without knowing any details for this structure. For example, the OS might try to pretend that all devices are either character or block devices and the structure may contain pointers to "read()" and "write()" functions and almost nothing else; or the structure might contain pointers to hundreds of different functions that may or may not be supported by the driver (e.g. pointers to "getKeypress()", "setVideoMode()" and "playSound()" functions that would all be set to NULL by a mouse driver that doesn't support them).

There are some problems with this approach of course. For example, the kernel might call the offset of 0x00000000 to a file which is believed to be a device driver. If this is not an actual device driver, an exception will happen. The kernel will then have to handle that exception and etc.

If it's not a device driver an exception may or may not happen - it could go into an infinite loop, or execute random instructions (including calling random kernel API functions) before crashing. I would assume you'd have some sort of binary header or magic number in the binary that the kernel can check to see if it actually is a driver.

Now, what if it's a malicious file specifically designed to look like a driver, that actually sends personal information to some scammer on the internet (like keypresses) instead? I'd assume you'd (for e.g.) put device drivers in a certain place in the file system so that the kernel can find them and so that other software (and normal users) don't have permission to write to that area.

Relocation as swapping them on and off the memory to disk storage and back? I am not sure what you mean by relocation in this case.

Relocation - using position independant code to have all device drivers running at different addresses in the same address space (e.g. like a monolithic kernel).

Will your kernel give each device driver it's own seperate address space (like my OS does) or will all the device drivers overwrite each other at the same addresses in the same address space?

[7] The kernel will then add this device driver to a queue. This will allow various device drivers to be hooked to a single device, running simultaneously.

About multiple drivers, my kernel will be charge of managing a queue of drivers in line to service various interrupts and IRQs. So once the driver is detected, the kernel will place it in the queue. When an IRQ is fired, the kernel will check for all device drivers that are supposed to handle that IRQ. Let's say it finds two of them. It will execute them in the FIFO manner. The second one should be noticed of the changes that the previously-loaded driver has made to the memory/IO/etc or unless it could possible duplicate the same thing the first driver has done. What do you think?

Imagine the first driver wants to send the floppy drive heads to track 0 (before doing a read or write) and the second driver wants to send the same floppy drive heads to track 55 (before doing a different read or write). Both drivers fight with each other and the floppy heads are continually moved from track 0 to track 55 and back. None of the drivers get anything done, until eventually the hardware fails (or the user notices, turns the computer off immediately and then refuses to run your OS ever again). Is this what you mean by "allow various device drivers to be hooked to a single device"?

How can several device drivers use the same device without stuffing each other up? Why do you want various device drivers to be hooked to a single device (is there any advantage that justifies all the synchronization and compatibility problems it'll cause)?

Alternatively, do you mean that the OS will support IRQ sharing (for e.g. where different device drivers control different devices that happen to share the same PCI IRQ line), and that different device drivers share the same software interrupt/s for their API? IRQ sharing is good (you need to support it for PCI devices to work properly).

IMHO different device drivers that share the same software interrupt/s for their API isn't good (it's a security problem, it's less efficient, etc). The kernel should handle the software interrupt/s and do security checks, etc before forwarding things to the correct device driver (not to all device drivers). In this case it'd be the kernel's API (not the device driver's API), where the kernel uses some other API to communicate with device drivers (possibly function pointers in the structure of information returned by the device driver's initialization).

The tricky part is that there's typically a hierarchical tree of device drivers, because there's a hierarchical tree of devices. For example, a floppy drive isn't connected directly to the CPU - the floppy drive is connected to a floppy drive controller that is connected to a "PCI to LPC" bridge that's connected to a PCI host controller that's connected to a memory controller that's connected to the CPU. This is especially important for power management (for example, so you don't send the PCI host controller to sleep while you're still trying to use the floppy drive controller).

This means that device drivers may use the kernel API (or the device driver API) to use other device drivers. For example, a file system might ask the floppy disk driver to do something, and the floppy disk driver might ask the floppy controller driver to do something, and the floppy controller driver might ask the ISA DMA driver to do something.

A device driver might handle all devices for a section of the hierarchical tree. For example, a single driver might handle the floppy drive controller and any floppy drives (and tape drives, zip drives, etc) that could be connected to the controller; instead of having seperate drivers for floppy drives (and tape drives, zip drives, etc) that all talk to a seperate floppy controller device driver. This may or may not be practical depending on the situation. For example, it might be fine for a floppy controller and devices connected to the floppy controller, but it'd be a nightmare for a USB controller and devices connected to the USB controller.


Cheers,

Brendan

[Craze Frog]

Relocation as swapping them on and off the memory to disk storage and back? I am not sure what you mean by relocation in this case.

Relocation is relocation. If you load driver a at position 0x1000000 then you can't load driver b at position 0x1000000 as well unless they have separate address spaces (which you didn't mention). So you will have to relocate driver b. However, flat binary files lacks relocation information, so you can't relocate it...

You can either:
- Use ELF or similar and relocate the drivers
- Give each driver a separate address space
- Use position independent code

[XCHG]

Brandon:

Your posts are really informative. I really appreciate your help.

Hi,
How do you load the disk driver from disk?

My kernel is loaded from the HDD in three stages:

• 1) The MBR loads the boot sector of the active partition
  2) The boot sector will load the extended boot sector.
  3) The extended boot sector looks for a file "/bin/kernel.bin" and loads that file into the memory.

The third step that is the Stage2 boot-loader has a minimal version of my File System and HDD Drivers built-into it. After thinking about it, I thought I, at the end of the day, had to have minimal support of HDD and file system somewhere that allows me to load files/kernel/etc from the HDD. So as the result, I will have a small built-in driver at boot-up that will load the HDD driver.

Why can't the device driver's initialization function allocate some memory (if necessary) and return a pointer to a structure of information (instead of having 2 initialization functions)? For simple devices, the structure of information might already be in the device driver's ".data" section (no memory allocation needed)...

My goal was to not allow the device driver to have any interaction with the kernel but allow the communication in the other way around. The kernel should ask for information from the driver. The driver should not be allowed to ask for anything from the kernel, not at least initially. This might not be a good idea though.

What will be the format for this structure of information? Will it be the same format (same fields, etc) for all device drivers (e.g. keyboard driver, hard disk, SCSI controller and video card)?

The format of this structure is somehow fixed but the length of each field is given to the kernel by the device driver. For example, the structure could be set like this:

Code: Select all

struct DeviceDriverInitializationStructure{
  unsigned int DriverVersionInBCD;
  unsigned int NumberofFunctionsSupported;
  void*        FunctionAddresses; /* As many addresses as that specified by [NumberofFunctionsSupported] */
  bool         HandlesIRQ;
  unsigned int HandledIRQNumber;
  bool         HandlesInterrupt;
  unsigned int HandledInterruptNumber;
};

The FunctionAddresses member will have its size in bytes equal to NumberofFunctionsSupported * sizeof(FunctionAddresses). So this could depend on the driver. My point here is that I don't want every single driver to have the same function called "Read" for example. The name of all the functions could also be exported in the driver's file format!

Will your kernel give each device driver it's own seperate address space (like my OS does) or will all the device drivers overwrite each other at the same addresses in the same address space?

Yes; I am planning on creating each driver its own virtual address space. They will all load at their virtual address space of 0x00000000.

How can several device drivers use the same device without stuffing each other up? Why do you want various device drivers to be hooked to a single device (is there any advantage that justifies all the synchronization and compatibility problems it'll cause)?

I believe you are right. After thinking about it more, I believe that the advantages (if any) could not possibly outweigh the hassle it could take to implement such scheme.

The tricky part is that there's typically a hierarchical tree of device drivers, because there's a hierarchical tree of devices. For example, a floppy drive isn't connected directly to the CPU - the floppy drive is connected to a floppy drive controller that is connected to a "PCI to LPC" bridge that's connected to a PCI host controller that's connected to a memory controller that's connected to the CPU. This is especially important for power management (for example, so you don't send the PCI host controller to sleep while you're still trying to use the floppy drive controller).

I had thought about this problem before. I had then come to the conclusion that it is best to allow a single driver to be in charge of everything that is related or could possible be related to a single piece of hardware.



Craze Frog,

Thank you for pointing this out. As I said above (as a reply to [Brendan]), I am going to have a separate virtual address space for each of the device drivers. They will be loaded into 0x00000000 in their virtual address space.

[Craze Frog]

Thank you for pointing this out. As I said above (as a reply to [Brendan]), I am going to have a separate virtual address space for each of the device drivers. They will be loaded into 0x00000000 in their virtual address space.

Ok, but loading them at position 0 isn't a very good idea because it makes all null pointers valid and that harms debugging.

[AndrewAPrice]

Why don't you use position independent code? Then then kernel can load it where ever it pleases. And for the module calling kernel code you could either use:
- interrupts, or
- when it calls the beginning of the module code to get the vtable/driver-info struct, you could pass another vtable representing kernel functions on the stack

[XCHG]

Ok, but loading them at position 0 isn't a very good idea because it makes all null pointers valid and that harms debugging.

You are right! Maybe some other constant entry point for every program? For example, if I load every driver at its virtual address of 0x0000FFFF, I could put some jump tables and etc at 0x00000000 - 0x0000FFFF. Or, I could simply load them at 0x00000004 or 0x00000008. How does that sound?

Why don't you use position independent code? Then then kernel can load it where ever it pleases. And for the module calling kernel code you could either use:

Yes, I really like the idea of Position Independent Code. In fact, I am looking for creating a scheme of loading library files just like DLL files in Windows. I believe DLLs in Windows also use the PIC technique.

[Craze Frog]

I believe DLLs in Windows also use the PIC technique.

No, they use relocation. The code is position dependent and additionally there is relocation information which says what parts of the program that needs to be fixed up if it's moved.

[Tyler]

I believe DLLs in Windows also use the PIC technique.

No, they use relocation. The code is position dependent and additionally there is relocation information which says what parts of the program that needs to be fixed up if it's moved.

Nothing says they can't be PIC. I have seen position independent DLL's before, but it's not always best, especially in code that needs to be as fast as possible and can't afford the loss base referencing causes it.

[Craze Frog]

Nothing says they can't be PIC.

It's not possible due to the way the loader works. Dll files MUST have a field in the header which says where it's linked for. The loader always loads it at that address unless something else is that address, then it always relocates it. So even if you put position independent code inside the dll the code will be relocated whether you want it or not by the loader.

Of course you can use PIC for certain parts of the code, but the dll itself will never be position independent.

Take a look in "Linkers and Loaders".

[Brendan]

Hi,

Nothing says they can't be PIC. I have seen position independent DLL's before, but it's not always best, especially in code that needs to be as fast as possible and can't afford the loss base referencing causes it.

The nice thing about PIC is that the same library can be at different starting addresses in many different address spaces (and in the VFS cache) while still using one set of shared pages. For Windows DLLs, if it's relocated it's modified and you need another set of pages (and you'd be better off statically linking to reduce memory consumption). Windows style DLLs are only "good" if relocation is unlikely.

IMHO (for my OS only), if you need speed then statically link and hope that the toolchain optimizes what you've got into what you want (e.g. remove all the dead code that's in the library that you don't even use). If it's likely that the library will only be used by one process or the library contains many small functions, then still statically link. Otherwise, turn the library into a service that runs in it's own address space and communicate with it via. asynchronous IPC so that the service and the process can both run on different CPUs at the same time to improve performance and scalability (and have a statically linked library of small functions to make it easy to use the service). No shared libraries and no DLLs means faster process startup (no linking, relocation, etc), and no dependancy nightmare (e.g. "I installed FOO and now BAR doesn't work")...

Note: this sort of thinking may not apply very well to other OS designs...


Cheers,

Brendan

[Tyler]

Nothing says they can't be PIC.

It's not possible due to the way the loader works. Dll files MUST have a field in the header which says where it's linked for. The loader always loads it at that address unless something else is that address, then it always relocates it. So even if you put position independent code inside the dll the code will be relocated whether you want it or not by the loader.

Of course you can use PIC for certain parts of the code, but the dll itself will never be position independent.

Take a look in "Linkers and Loaders".

The Ability to Relocate is simply another feature. Nothing stops you from having an empty, or small set of relocations and making the rest of the code Position-Independent. It is no more than adding the extra ability to relocate to ELF and then not using it.

I'm not speculating here... i have compiled PIC in PE's for the purposes of speeding up load times when stream lining execution isn't so important. Even then that only matters for incredibly tight loops, so PIC is often the best option.

[Tyler]

Hi,

The nice thing about PIC is that the same library can be at different starting addresses in many different address spaces (and in the VFS cache) while still using one set of shared pages. For Windows DLLs, if it's relocated it's modified and you need another set of pages (and you'd be better off statically linking to reduce memory consumption). Windows style DLLs are only "good" if relocation is unlikely.

IMHO (for my OS only), if you need speed then statically link and hope that the toolchain optimizes what you've got into what you want (e.g. remove all the dead code that's in the library that you don't even use). If it's likely that the library will only be used by one process or the library contains many small functions, then still statically link. Otherwise, turn the library into a service that runs in it's own address space and communicate with it via. asynchronous IPC so that the service and the process can both run on different CPUs at the same time to improve performance and scalability (and have a statically linked library of small functions to make it easy to use the service). No shared libraries and no DLLs means faster process startup (no linking, relocation, etc), and no dependancy nightmare (e.g. "I installed FOO and now BAR doesn't work")...

Note: this sort of thinking may not apply very well to other OS designs...


Cheers,

Brendan

I had considered that before. I knew that you weren't allowing Shared Libraries for the purpose of speeding up load's and because of the distributed nature of your concept. I didn't think you would try the services idea though, it is an interesting mix of protection and sharing, but i assumed you would see more advantage to single address space processes with the current hardware support.

I can see why it would be useful if the majority of Libraries could be statically linked without worrying about excessive increase in file size and memory usage, but the modular design of my current system would make the overhead of that much IPC too great. Most software and libraries only exist to service other's in order to try to increase usage, my way of doing the very opposite of using more static linking.

Personally, i'd much rather waste a little Hard Drive space having a copy of shared libraries that require constant relocation, already relocated to a suitable address. The real problem is that when i have tested large number's of services with IPC is that the slow down is noticable after a few more librariers than the average and becomes more drastic with every new system.

Do you have some ingenius plan for larges numbers, or really expect it to be only a last resort to a usual method of static linking?