Device drivers

[bzt]

Hi,

It's amusing how confused you all became just because of the function names. The answers are clear: you're talking about a DIFFERENT system which happens to have similar names.

@StudyCaps: the critisism are not about this interface, but about something else, most notably complex frameworks, Linux ioctl and file abstractions, neither of which play part here (which makes me think none of you gets the concept of ODD).

@linuxyne: this is NOT a framework, just a standardized interface for low level drivers. And for the Nth time, there are no files involved. There's a huge difference when fd selects a context outside of the driver's scope, and when index selects a context within the driver's scope. The former context is kernel specific, while the latter is purely driver specific.

@Korona: "it would be possible to first focus on what drivers provide" you are starting to understand the concept! And not just at first, standarizing what the driver provide is all what ODD is about, nothing more. The memory allocation is required because the driver must handle it's internal device context. Read/write/ioctl should operate on one context at a time, true, but attach/init must allocate (and detach/fini free) memory for that device context. I've suggested to make it a macro so that a monolithic kernel (without libc malloc) can map it's kernel heap memory allocator for the driver. The final version going to have only one odd_realloc(), with two restrictions: with NULL pointer it should work as malloc(), and with size of 0 as free().

@Strum: if a driver looks completely different (it does not get data from the device or send data to the device) then it's disfunctional. Btw ODD would work pretty well with your scheme, you should return an ODD read/write function address or an ODD function wrapper address; and the other parts of your kernel doesn't need to know which drivers are ODD compatible and which aren't. I've specifically taken into account the scheme you have.

@alexfru: you are not talking about ODD, but about Linux ioctl. Don't let the name fool you.

Cheers,
bzt

[alexfru]

Where are the files in my proposed interface? You are biased by the names

@alexfru: you are not talking about ODD, but about Linux ioctl. Don't let the name fool you.

Your interface can't seem do anything other than moving data in one direction or another independently (as with files). It doesn't appear to offer anything for a synchronous device (e.g. ADC/DAC), which produces and consumes data simultaneously at the same rate. There are applications where these two streams of data must not go out of sync (e.g. echo cancellation, modems) and so there needs to be a way to call the driver and exchange N output samples for N input samples or something to that effect (you can't just drop or insert a buffer in one of the streams without balancing that with the other stream).

Your interface is very simple. But I think it's too simple for real world applications. How do I implement a NIC driver? Do I just make the driver that can only fill and drain the circular buffers of network packets? What about the link, any API for link control? What about the myriads of options that you can see in e.g. ixgbe drivers? Anything for performance like directly mapping the circular buffers to wherever desired? I mean, you can squeeze all of it in an ugly way into the set of methods offered by the API heavily using IOCTL for what doesn't fit into others. You can also split each complex device logically into a set of pretend-separate devices and organize them into a tree or graph such that there would be a link-control subdevice with meaningful read/write methods just for that functionality and data-only subdevice that would just fill and drain those circular buffers and other subdevices... That's not making things simpler, I'm afraid.

I'd think the API should reflect behavior of devices of different kinds and offer sufficiently rich and comfortable ways of working with them. If it only offers a abstract way of transferring some data to/from somewhere without taking into account much or any of device/medium specifics, it's just an API without a purpose (but it's good enough for files, we know that!:) ).

What's your point? Simple API for crippled (by design) drivers? Limited API that must be abused in order to get desired functionality and/or performance? Deceptively simple API that suddenly unfolds its complexity by requiring trees of fake subdevices? I'm still not seeing the point of how this all is supposed to be helpful. Really.

[Octocontrabass]

It's amusing how confused you all became just because of the function names. The answers are clear: you're talking about a DIFFERENT system which happens to have similar names.

You haven't told us much besides the function names. You can clear up the confusion by writing a more detailed API proposal.

With that said, I have some questions about your API. What's the point of the seek() function? What purpose does it serve that can't be handled by implicit seeks during read() and write() calls? What purpose does it serve that can't be handled by ioctl() with a "seek" parameter?

How does your API handle hardware that may service requests out-of-order, such as SATA NCQ or NVMe?

[linuxyne]

@linuxyne: this is NOT a framework, just a standardized interface for low level drivers. And for the Nth time, there are no files involved. There's a huge difference when fd selects a context outside of the driver's scope, and when index selects a context within the driver's scope. The former context is kernel specific, while the latter is purely driver specific.

Your OP said you want "common, reusable driver set" which is "source-compatible" across different systems. I don't care how you twist and turn your requirements - each OS will need a framework to provide a workable common "interface" to run a source-compatible driver.

Not only that, you want *all* drivers on *all* participating systems to conform to a single standard. If you look at the drivers for Linux and BSDs (or between BSDs), that might indeed be the case - those drivers might have very superficial changes, while the bulk of the logic might follow a common, consistent pattern. And in a real world, even that's enough for practical purposes.

I would not mind spending a day or a week (depending on the complexity) touching up the changes, if that's all it takes to port a driver from one system to another. That saves around 5-6 OS groups from maintaining a common framework being proposed. Instead of implementing a common framework across 5-6 different system groups, it might be easier to pick one OS of our choice and spend effort towards making it better than the rest so that the rest will automatically sink and wither away.

Again, fd/index/FILE all identify the device and help in locating/maintaining the device's state.

It must be amusing that Microsoft maintains two different frameworks - one for network devices and one for storage. You show them folks at Redmond how its properly done!

My advice at the end of my last post was serious - take two OS and implement (at least v0.1) what you speak about. I think that's the best way to show the world how stupid it is and how smart you are (since that seems to have become your goal; not that such implementations are supposed to be normally abused by their implementors to show ones personal superiority). Seriously, all else you talk on this topic is hot air, which is indeed amusing.

Edit: You do know that FUSE, on which you base your idea, needs each filesystem to conform to a standard which is described by both a userspace library and a kernel module? If I want my fuse filesystem to run on both Windows and Linux, I expect each OS to provide their /own/ implementations of the library and the module. The library and the module /is/ the framework, which exposes the common API under whose spec one writes the filesystem.

[bzt]

Hi

I see no point in further explaining the concept since you haven't read and understund my first post so far. Maybe you just do not have the required knowledge to comprehend the concept?

@alexfru: "How do I implement a NIC driver?" If you have to ask that question then a standardized interface can't help you, no doubt. I'd suggest to learn more on file abstraction too, because you clearly don't know what that is. May I suggest to do some research on VFS first perhaps?

@Octocontrabass: "You haven't told us much besides the function names." Please read again the OP. Until you get what's written so far, there's absoultely no point in explaining the concept any further.

@linuxyne: "you want "common, reusable driver set"" Nope, read again my OP. "You do know that FUSE, on which you base your idea" I did not based my idea on FUSE, that was just an interface example. Isn't that obvious? Read the OP. And according to fd/index, read my previous post. I won't repeat myself, as that won't make any good for you. Let's talk when you've understud what I wrote so far.

I had a little hope when Korona seemed to understand the concept, but clearly he is a minority. Most of you are unable to read and comprehend my answers, therefore it is futile to explain the concept any further, at least not on this forum. I wonder how you understand and implement other specs (no offense, but you are misinterpreting and totally forgot about key parts of my OP).

Cheers,
bzt

[linuxyne]

@linuxyne: "you want "common, reusable driver set"" Nope, read again my OP. "You do know that FUSE, on which you base your idea" I did not based my idea on FUSE, that was just an interface example. Isn't that obvious? Read the OP. And according to fd/index, read my previous post. I won't repeat myself, as that won't make any good for you. Let's talk when you've understud what I wrote so far.

Great. You need an interface; it needs a framework, else what are you going to achieve with just an interface without anybody implementing the framework and the drivers corresponding to them?

Do you only care about bridging monolithic and microkernel OS? If so, clarify that.

I don't mind repeating myself. The fd/index/FILE are indistinguishable properties.

FUSE: is a good working example on how such a source compatible interface can be done

This is more than enough for me to know that that wants to mimic what FUSE did. The word "done" is to be taken as "implemented". It is also
marked as "working" which is another positive.

Another example: Vulkan is just an interface (like yours is just an interface), but there are concrete pieces of code surrounding this API which do the actual work of realizing the goals that the interface set out to achieve.

Also take a look at Linux's USB stack.

I had a little hope when Korona seemed to understand the concept, but clearly he is a minority.

Awww... This is quite cute. That aside, drivers demand access to kernel functions by calling/linking with them. Drivers expose their interfaces through their export tables. Can each OS not implement its loader to recognize how it needs to patch-up/shim a driver?

Most of you are unable to read and comprehend my answers, therefore it is futile to explain the concept any further, at least not on this forum

That's because you are not a good teacher. I somehow believe that you take pride in that fact.

Why do you need somebody else's approval before you can implement your interface? Show the work, and people will join as they understand what is being built. Honestly, you are selling the world short, and that only exposes your arrogance.

How does this forum's inability (so you say) to understand what you are proposing, preventing you from designing and implementing even a tiny bit of your proposed scheme on your own? Walk the talk.

[Octocontrabass]

@Octocontrabass: "You haven't told us much besides the function names." Please read again the OP. Until you get what's written so far, there's absoultely no point in explaining the concept any further.

So, what you're saying is read() and write() depend on the state from the previous read()/write()/seek() call instead of being atomic, so some form of delegation will need to be implemented if the driver will be used by multiple callers. On top of that, when you're accessing the device's data in a nonlinear format, you'll need a lot of seek() calls and suffer the associated overhead.

Since the caller can only make one request at a time, there's no way for hardware that's capable of servicing requests out-of-order to do so.

Have I gotten everything you've written so far?

[linuxyne]

"To sum it up, it's a C/C++ header file to provide a standardized driver API and interface to make porting drivers among hobby OSes easier. Nothing more, nothing less."

Oh, you are going to need more than just a header. For e.g. a framework in each participating OS .

I think this is the case of NIH syndrome. Or, bzt has just discovered, for himself, that driver frameworks are a thing.

The obviousness of the project, pointed out though it was quite early by others, was buried under the trifling details of "files" and "frameworks", blown out of proportions to match the OP's understanding.

Systems abound plentifully with various driver frameworks. Windows gives its driver frameworks a cryptic name: Windows Driver Frameworks. Linux has frameworks built upon lower-level frameworks, such as the USB stack. People seem to have written wrapper frameworks over Vulkan to ease driver development.

I think it is beneficial to write drivers for commercially available systems, before attempting to write one's own OS.

[StudlyCaps]

bzt just invented the block device abstraction, we're all just too dumb to understand it.

[DavidCooper]

I think the way forward with this would be to write a single device driver demo to illustrate the concept and see how many people are able to integrate it into their OS. Any who can't do so because of inadequacies of their OS or their inability to wire it all up won't matter - what will prove the concept is if a number of OSes of radically different design are able to use that driver without needing to change the way they work. For some devices, it may only be practical to manufacture a set of components for a device driver and leave the OS designer to write the missing ten to fifty percent needed to turn them into a working driver, but that would still be a lot easier than having to write every driver from scratch. This would be more use than having to warp your OS to a specific design of universal device driver which may prevent you from building an OS that works the way you want it to: I want device drivers to conform to my OS design and not the other way round, so I'd be interested to see what you come up with.

[alexfru]

I see no point in further explaining the concept since you haven't read and understund my first post so far. Maybe you just do not have the required knowledge to comprehend the concept?

@alexfru: "How do I implement a NIC driver?" If you have to ask that question then a standardized interface can't help you, no doubt.

I can see a communication problem here. On one hand you're saying it's a simple API, on the other it appears to be for a very narrow audience (yourself and a few other geniuses) since we can't grasp what it really is from your original post and the follow-ups. And then you're giving up on explaining it to the mere mortals.

If there really is an idea and something useful in it, you have so far failed to express it clearly enough if at all.

You need to do a better job to win our minds. Explain stuff, don't just say/imply we aren't smart enough. We aren't, but I hope that's not the reason this thread exists.

P.S. I have read your original post and some of the responses multiple times. If there was something significant that I missed, it probably needed some emphasis or elaboration.

[bzt]

Hi,

It is definitely not my job to teach you the difference between an API interface, a device framework, files and indeces etc. If you think that, then you're mistaken.

Btw, I have never said you are not smart, you did. All I said was you're not experienced enough to comprehend what I'm saying. (Just to clearify, the former implies you can't learn, the latter implies you can, you just need more time.) And also you should practice interpreting and understanding specifications, because all of you misinterpreted or simply forget key parts of what I've written so far.

Let's talk about this once you have the required knowledge.

Cheers,
bzt

[alexfru]

And also you should practice interpreting and understanding specifications, because all of you misinterpreted or simply forget key parts of what I've written so far.

Well, your informal specification doesn't appear good or showing all of the key things that you deem important (since we're missing them).

Let's talk about this once you have the required knowledge.

Let's talk about this when you can express your thoughts clearly and communicate effectively.

[linuxyne]

It is definitely not my job to teach you the difference between an API interface, a device framework, files and indeces etc. If you think that, then you're mistaken.

The fact that you still believe that the differences are relevant exposes your lack of understanding. The fact that you believe that an interface alone is sufficient for what you want to achieve, shows your lack of experience and forethought.

I can cite even more examples from your past conversations, but that will be for another thread.

Btw, I have never said you are not smart, you did. All I said was you're not experienced enough to comprehend what I'm saying. (Just to clearify, the former implies you can't learn, the latter implies you can, you just need more time.) And also you should practice interpreting and understanding specifications, because all of you misinterpreted or simply forget key parts of what I've written so far.

Let's talk about this once you have the required knowledge.

And you should do your research before trying to reinvent things already made, should learn when one needs to speak in general terms and when in gory details.

For e.g., all you had to do in your OP is to point at Windows Driver Frameworks and tell people that you want hobby OSes to implement that at a manageable scale.

You do have a communication problem, which is only worsened by your denial of its existence, and planting false accusations of "lack of knowledge" on others. I can see why someone might lose patience with you.

You need to go to college and take a class in logic/discrete maths/theory of computation to get your logic straight. You are not, as you very likely believe, an advanced developer.