Alternatives to File System as a means for OS interaction

[jayjay]

Hi all,

I'm currently working on the file system and driver interface for my kernel, and have been wondering if there is any reasonable alternative to using a virtual file system to present devices and kernel constructs to user space. I'm sure we're all familiar with /proc, /dev and /sys, but is attaching these systems to the file system really the best solution?

The way I see it, the pros are:

• Simple for users to understand and interact with - same as navigating the file system
• Rigid and well tested interface
• Built-in permissions
• A single, unified interface for all systems to interact through.

But this does not come without any cons:

• Implementation specific arguments to generic API/sys calls - ioctls are specific to the device, and even driver, for example.
• Not all structures fit well within a file system structure - cyclical references, which can be found quite often in sysfs, can cause all sorts of problems.
• The entire interface does not need to be present at all times - excessive files would get in the way of understanding, however the same could be said for minimal file use, or a stateful implementation.

This is just the tip of the iceberg, but I am interested in your opinions. What do you think the pros/cons of attaching internal kernel constructs to the file system are? Are their any reasonable alternatives? And most importantly, How do you tie your kernel internals to user space?

Thanks,
Jay Jay.

[max]

Hey Jay Jay,

that's an interesting topic. As you said, using the filesystem has advantages as it's very generic. Though I think that is somehow "mis-used" for that, I think a communication via a central server & pipes would be more suitable.

One of my ideas how to solve it is the following. First, a device - say, a printer - is installed to the system. The driver for that device establishes a connection to a central "device management server" (the DMS) that keeps track of all installed devices. The DMS knows a list of generic functionalities; for example a printing functionality, an audio input functionality etc.. The device driver tells the DMS about which of these functionalities are supported.

A program can now ask the DMS which devices are available for generic-function-x, and the DMS supplies the information necessary to communicate with that devices driver. The driver must then react to the generic commands (for example via a pipe with a defined protocol) and make the device do the thing.

The information in the management server can then also be used to for example display in a user device management center which devices are installed and what is available. If a device has a non-generic functionality, the driver can still register it at the DMS; a program must then know how to communicate with the device driver.

The advantages of this approach are, that for common devices the generic functionalities (which should be versioned, to allow steady improvements) are easy-to-use for programs and should be well implementable for the drivers. Also, the management server could be simply exchanged as it runs in userspace.

These were just my thoughts so far, and I don't know yet if it is a practical approach as my only drivers are mouse and keyboard so far

Greets,
Max

[jayjay]

Hey Jay Jay,

that's an interesting topic. As you said, using the filesystem has advantages as it's very generic. Though I think that is somehow "mis-used" for that, I think a communication via a central server & pipes would be more suitable.

Is this not what the file system provides? named pipes to a backend controller? What I'm interested in is the removal of pipes, which ultimately means pure syscalls returning populated structures - but is there an alternative to this too?

One of my ideas how to solve it is the following. First, a device - say, a printer - is installed to the system. The driver for that device establishes a connection to a central "device management server" (the DMS) that keeps track of all installed devices. The DMS knows a list of generic functionalities; for example a printing functionality, an audio input functionality etc.. The device driver tells the DMS about which of these functionalities are supported.

A program can now ask the DMS which devices are available for generic-function-x, and the DMS supplies the information necessary to communicate with that devices driver. The driver must then react to the generic commands (for example via a pipe with a defined protocol) and make the device do the thing.

This sounds like an Entity-Component-System architecture, which would make for interesting design. Treating modules/drivers as an entity with attached components, and each component is driven by a centralised system dedicated to the features of the component. So you could have a UART component, which is derived into the specific UART types, and is driven by the UART system, with data being provided by another component, or to another component (for example USB-to-TTL devices). Drivers could chain or combine components to match the features of the device.

The information in the management server can then also be used to for example display in a user device management center which devices are installed and what is available. If a device has a non-generic functionality, the driver can still register it at the DMS; a program must then know how to communicate with the device driver.

The advantages of this approach are, that for common devices the generic functionalities (which should be versioned, to allow steady improvements) are easy-to-use for programs and should be well implementable for the drivers. Also, the management server could be simply exchanged as it runs in userspace.

But, ultimately, this system would still depend on a pseudo file system due to the requirement of pipes - to access the pipes, you open them by name on the file system.

This does remind me of the way some Windows devices present in user space - COM1, COM2 etc. can be opened by name, but do not technically exist within the file system (but still act as files nonetheless).

[tlf30]

Hello Jayjay,

I have been working with a system lately that might be of interest.

I am working in a modular base OS that does all of its memory and communications in a separate module (MC Module) from the kernel. The kernel is only responsible for direct hardware communication, and a separate module is used to create the execution environment and tasking system.

Because of this, we handle device communication (COM ports, USB, ATA, PCI, NICs) in a two system process. The kernel has 'controllers' that provide a standard interface to device's buffers. Then the MC module has 'managers' that links to the controllers. In doing so, the software on the system requests a new io buffer from the manager that it needs (COM Manager, NIC Manager ...), and the manager transfers the buffer to the correct controller where the controller will handle the buffer and translates it to the hardware's understanding.

This allows us to perform IO functions on devices much like service calls in the OS. We have no entries, or relations to the filesystem.

Just my two cents,
Trevor

[SpyderTL]

There was a similar post about a related topic yesterday. You may want to read through it, as well.

http://forum.osdev.org/viewtopic.php?f=15&t=29423

As I mentioned in that thread, there have been several threads over the years discussing different file system concepts. (Object based, database table based, etc.)

Personally, I'm sticking with plain old "objects" and "classes" to handle all of my physical (cpu, hard drive, sound card, etc.) and non-physical (files, folders, network connections, etc.) resources in my OS. I'm really trying to avoid using "traditional" solutions to these types of problems, and rethinking a lot of concepts and terminology that most people just take for granted.

The way I see it, as soon as you introduce more specific structures to your system (Pipe, Thread, File, Handle, Port, Stream, etc.), they are no longer "interchangeable", and you must write specific functions and utilities for each of them. If you want to write debug information about a Pipe, you have to write specific code that understands your specific Pipe memory structures. If you want to write debug information about a Thread, you have to write even more code. And so on...

Treating all "objects" the same at least gives you a reliable platform that you can use to add functionality to ALL of the components in the system at the same time.

[carbonBased]

In my OS, drivers are object files which reside in ring 3. They expose a mapping of strings/paths to functions (which can be mapped to a filesystem, if you wish).

e.g., a simple text console might expose:

/resolution/get
/resolution/set
/cursor/get
/cursor/set
/buffer
/buffer/currentCharacter

Or something similar (I haveen't really worked out a standard naming scheme). Reading/writing to these would map to calling a function within the driver.

Still fairly filesystem-esque, but would allow a program to pre-lookup and cache the items it was interested in, and call them directly as required (which could be implemented as a single method call to a user-land mapped object).

The power of mapping it over the filesystem means that shell scripts can do the same thing.

[SpyderTL]

Still fairly filesystem-esque, but would allow a program to pre-lookup and cache the items it was interested in, and call them directly as required (which could be implemented as a single method call to a user-land mapped object).

The power of mapping it over the filesystem means that shell scripts can do the same thing.

This is actually pretty similar to what I'm doing, although not just for drivers. The entire OS is made up of run-time classes that can be looked up by namespace. (Right now, since I'm using XML, my namespaces are URIs. But they could really be any unique string.)

An application can find a reference to a class by passing its namespace, and it can hang on to that reference as long as it wants. Then it can find a reference to a method (or field) on that class by name, and it can store that as well.

All classes have one or more "static" methods that will create an object of that type (allocate memory, initialize fields, etc.) and return a reference to the object, which act as "constructors".

Using this simple framework, I should be able to model just about any functionality an application might need -- devices, streams, threads, locks, codecs, services, etc.