Unified Driver Interface

[Brendan]

Hi,

I know what you are aiming at, Brendan, and indeed I don't think it is possible to create a generic driver interface e.g. for 3D-accelerated gfx cards.

That falls well short of the mark - I don't think it's possible to create a generic driver interface for any single specific type of device, much less for all different types of devices at the same time.

But UDI is a functional specification (I just don't know enough about EDI to comment), and if "all" those generic approaches give to the OS dev community is a common set of drivers for hard drive controllers, serial and USB ports, and network cards, that's nothing to be sneered at IMHO.

You're right - maybe I should have started with something like serial ports (instead of video cards), so that we can see just how badly the "generic driver interface" concept fails, even with very simple cases...


Cheers,

Brendan

[Solar]

Erm... you seem to claim that it cannot be done, yet UDI exists, both in framework and in drivers, in reality?

[Kevin]

Do you have an idea what it takes to implement UDI in one of our OSes? At a first glance it looks fairly complex. It might be the "real" thing, but I think many people are rather looking for something simple (i.e. doable on a weekend) that still gives them basic hard disk/network/graphics/whatever support.

Anyway, we'll see what you present on that website, maybe it's easier than it appears.

[Brendan]

Hi,

Erm... you seem to claim that it cannot be done, yet UDI exists, both in framework and in drivers, in reality?

Um.

SCO started "Project UDI" and were the main advocates of UDI. Project UDI ported UDI to other OSs, and the other OSs ignored it or outright rejected it. After that (around the year 2000), it mostly became SCO's proprietory driver interface, and SCO gave up trying to convince other OSs to use UDI, and started suing everyone instead. The latest news is that SCO sold its UNIX division to someone called "Gulf Capital Partners" last month (June 2009) as part of a bankruptcy liquidation garage sale (I think they sold some old cassette tapes, assorted used kitchenware and some second-hand children's clothes at the same garage sale).

It's not exactly what I'd call a success story.

Despite all of this, IMHO UDI has never qualified as a generic device driver interface for all OSs, even though at one time it may have qualified as a generic device driver interface for Unix clones, Unix work-alikes and other reinventions of the same old wheel.


Cheers,

Brendan

[Love4Boobies]

Sorry, I've been a bit absent for a while but I feel I need to say a thing or two about UDI since I'm familiar with it.

@Brendan: Interesting note but the failing history of UDI isn't related to the piece of technology it represents IMHO.

Kevin said that UDI is far too complex to implement. But is it, just because its specification has 3 core volumes? I have played a little with UDI and even used the reference implementation to port the environment to my OS. Not only that, but with the headers they provide, I also wrote a few drivers that have been tested not only under my OS, but also Linux (yes, I'm aware the environment is not part of the git tree but it was ported by STG. UDI environments can also be found for Mac OS, FreeBSD, Solaris, UnixWare and others).

Like Solar said, UDI is a framework. If you take a look at the specification, you will find a lot of macros, types, etc. that it defines. It's like saying "hey, I can't use C, did you see all the functions they provide in the standard library?". You can use the UDI framework or rewrite it just like you can use the standard library that comes with the compiler or write your own.

Why is UDI good? Compatibility is vital - I don't have time to write hundreds of drivers for every piece of hardware I want to support. Do you, Brendan? Ok, this doesn't make it good, it just makes it useful. Now let's take a look at what makes it useful: it's synchronization model for MP scalability is worth mentioning. Benchmarks have shown increased performance over legacy drivers such as DDI drivers (and not just on MP systems). I've seen more than enough driver interface bottlenecks around to know when something like UDI hits the right spot. UDI drivers are also easy to develop since the design actually eliminates the risks for some common bugs (at least, partly) like deadlocks.

How many of you support the Multiboot specification? Do you think that is any good? On the other hand, UDI actually is if you take the time to have a close look. How many of you have even read the UDI specification?

If I sound a bit tough in the text above, I'm sorry. It's wasn't my intention at all.

[Brendan]

Hi,

Why is UDI good? Compatibility is vital - I don't have time to write hundreds of drivers for every piece of hardware I want to support. Do you, Brendan?

If I write an OS that has a huge number of device drivers and works perfectly, but doesn't include features that other OSs don't have, then I've failed miserably and wasted my time. On the other hand, if I write an OS that does include features that other OSs don't have, but doesn't support many devices and only works properly some of the time, then I've got something worthwhile - I've successfully provided reasons for people to be interested in my OS.

Not supporting many drivers is an implementation issue - it's easy to fix later, without changing any existing code. Not having unique features is a design flaw - it can take a full "redesign and rewrite from scratch" to fix that in some cases.

Do I have time to write hundreds of device drivers? I doubt it; but I only really need about 15 drivers to demonstrate my OS's potential - serial, parallel, PS/2 keyboard, PS/2 mouse, USB keyboard, USB mouse, a generic USB storage device driver, ATA/SATA, one good video driver (Intel's graphics probably), and a few ethernet card drivers. That's entirely possible.

Can I afford to use UDI? If was willing to accept UDI there wouldn't be any reason for me to write an OS in the first place. I'd probably write a 3D engine instead.

How many of you support the Multiboot specification? Do you think that is any good? On the other hand, UDI actually is if you take the time to have a close look. How many of you have even read the UDI specification?

I support Multi-boot, but only as an optional/legacy boot loader (I've got my own native boot loaders); and I use multi-boot in my own way - e.g. one of the first thing my multi-boot code does is switch back to real mode so that it can do things properly, and multiboot is only really used to get some files into RAM.

I've scanned through the UDI specification (but didn't really read it properly), mostly to check if it covers anything that I hope will become unique features of my OS. Fortunately it doesn't, so my current plans are safe.


Cheers,

Brendan

[Kevin]

Kevin said that UDI is far too complex to implement. But is it, just because its specification has 3 core volumes?

What I said is that it appears to be complex at the first glance. I don't feel qualified to judge that, I haven't even started to really read it (because I don't see a pressing need to implement it). I'm just waiting what will appear on your UDI page.

I have played a little with UDI and even used the reference implementation to port the environment to my OS. Not only that, but with the headers they provide, I also wrote a few drivers that have been tested not only under my OS, but also Linux

Great! So could you actually tell me what effort it took you to implement it in your OS? Is your support complete or just enough for some specific driver?

[Love4Boobies]

@Mods: can you guys split this thread in 2? It's been a long time since we've been talking about UDI instead of EDI here.

I support Multi-boot, but only as an optional/legacy boot loader (I've got my own native boot loaders); and I use multi-boot in my own way - e.g. one of the first thing my multi-boot code does is switch back to real mode so that it can do things properly, and multiboot is only really used to get some files into RAM.

Aha! So there you go. Supporting UDI is again a compatibility thing. You can support multiple driver interfaces at the same time just like you can support multiple boot interfaces. There aren't much drivers out there to make it attractive unfortunately, but there are a few. An UDI repository is currently being worked on (framework, driver database, documentation and perhaps some tutorials later on).

Great! So could you actually tell me what effort it took you to implement it in your OS?

The effort wasn't that big. After taking the time to read the specs I was actually amazed how easy it was to port the UDI environment.

Is your support complete or just enough for some specific driver?

Well... as I mentioned before, I used the UDI framework. Unfortunately, since the project has been declared defunct, they never got to finish that framework and don't have 100% compliance to the specifications. However all UDI OSes I've seen so far implemented the environment and drivers using their framework so I couldn't find anything that didn't work for me.

I do hope I will have some spare time to put a bit more work into their framework but I'm sure I can't continue work on environments for Solaris, Darwin, Linux, etc. My own OS is enough. If after gathering some drivers in the repo (hold your horses, the project only begun just now - if anyone's interested, it was started by Solar and myself) it will become more popular maybe some of you will want to use them and thus implement UDI too. If maintainers can be found for the above environments (and others), it'd be great but that's probably a pipe dream.

[Brendan]

Hi,

I support Multi-boot, but only as an optional/legacy boot loader (I've got my own native boot loaders); and I use multi-boot in my own way - e.g. one of the first thing my multi-boot code does is switch back to real mode so that it can do things properly, and multiboot is only really used to get some files into RAM.

Aha! So there you go. Supporting UDI is again a compatibility thing. You can support multiple driver interfaces at the same time just like you can support multiple boot interfaces. There aren't much drivers out there to make it attractive unfortunately, but there are a few. An UDI repository is currently being worked on (framework, driver database, documentation and perhaps some tutorials later on).

I can support multi-boot without it effecting any code that's used after boot. I even use exactly the same second stage boot code for both multi-boot and native boot loaders.

To support UDI, I'd need to create a wrapper around each UDI device driver for each type of device (to turn it into a process that understands my messaging protocols). Also, instead of being able to say "all device drivers can be used directly by all emulators/virtual machines" I'd need to add a "does/doesn't support virtualization" flag in the device driver executable's header, which means that lazy programmers that don't feel like supporting virtualization in native device drivers could get by without supporting it - maybe out of 100 drivers I'd end up with 2 UDI drivers and 23 native drivers that don't support virtualization, and only 75% of drivers will work with virtualization (rather than 100%). Then there's other implementation issues (does UDI support I/O priorities, does UDI expect to tell the OS which resources it needs or can it be made to work where the OS tells the driver what resources it uses, will the UDI driver free any RAM used for caches/buffers when the kernel asks it to, etc) - I don't know the answers to these sorts of questions and I don't know what other issues I'd face; but I'd assume there's at least a few more problems I'd need to work-around.

I went looking for UDI drivers. In the STG "UDI for Linux Reference Environment" I found:

• one driver for DEC 21140/21143 based ethernet cards
• one device driver for CMOS
• one device driver for some Adaptec SCSI controllers (6 versions of DPT SmartCACHE/SmartRAID controllers)
• some useless pseudo/test/example code
• one device driver for a "Guinea Pig" - I don't know if this is a real device or more useless pseudo/test/example code

For all of these the copyright is the same, and it looks like the code came from Project UDI back in the year 2000/2001 and hasn't been touched since.

I couldn't find any other UDI drivers of any kind. And if this list is a complete list of all UDI drivers then (if I ignore the other problems) it's going to cost me more time to support UDI than it'd save me.


Cheers,

Brendan

[Solar]

This is probably the first time I actually disagree strongly with Brendan, but...

I don't know the answers to these sorts of questions and I don't know what other issues I'd face; but I'd assume there's at least a few more problems I'd need to work-around.

...that just sounds too much like "not invented here syndrome".

[Brendan]

Hi,

...that just sounds too much like "not invented here syndrome".

That just sounds too much like "let's all give up and use Windows/Linux syndrome".

If I wrote another Unix clone (rather than joining one of the existing very good projects), do you honestly think anyone would care about my OS if/when it's finished?

If I wrote a Windows clone (rather than joining ReactOS, which at least has some hope), do you honestly think anyone would care about my OS if/when it's finished?

If I wrote a VMS clone (rather than joining OpenVMS), or a BeOS clone (rather than joining Haiku or Zeta), or an AmigaOS clone (rather than joining MorphOS, AROS or AtheOS), or any other type of clone, do you honestly think anyone would care about my OS if/when it's finished?

If my OS is not different to all other OSs in one (or hopefully more) major ways, then my OS is a pointless waste of time.

Now, how can I make my OS different when all of the drivers are generic/UDI drivers that have the same features that almost every other OS has? It simply isn't possible.


Cheers,

Brendan

[Troy Martin]

Deep stuff, Brendan.

I agree with both of you. A single interface for drivers would be cool (that way we can work to better each others' projects while still contributing to our own), but it does have a possibility to kill the uniqueness of a project.

*vanishes*

[gravaera]

Okay: for me, it's just a matter of what benefits I gain from the driver interface spec. And it seems so far from what I've read, that UDI is the thing for me.

[01000101]

If I wrote another Unix clone (rather than joining one of the existing very good projects), do you honestly think anyone would care about my OS if/when it's finished?

If I wrote a Windows clone (rather than joining ReactOS, which at least has some hope), do you honestly think anyone would care about my OS if/when it's finished?

If I wrote a VMS clone (rather than joining OpenVMS), or a BeOS clone (rather than joining Haiku or Zeta), or an AmigaOS clone (rather than joining MorphOS, AROS or AtheOS), or any other type of clone, do you honestly think anyone would care about my OS if/when it's finished?

If my OS is not different to all other OSs in one (or hopefully more) major ways, then my OS is a pointless waste of time.

I honestly couldn't agree more. If it's no different from anything else, then why would someone even take a second look at it? Unfortunately I don't think I'll enter the main debate here as either one of you would crush my thoughts on the subject .

[Love4Boobies]

But UDI itself doesn't make it impossible for you to add extra features, that's one of my points. It's really flexible. In the words of Robert Lipe, one of the guys who used to work at SCO's operating systems and UDI (and who I contacted recently):

I think there's some really great engineering in there and I can't count how many times I've read an announcement of some development in an OS and realized that UDI solved that problem nearly ten years ago - and we never really got to explore some of the innovation that UDI alowed. Having invested so much soul into UDI, I'd be delighted to see a new wave of interest in it.

What are these great features you want your drivers to have and are incompatible with UDI?

I have also recently found that there have been a lot of UDI drivers out there, unfortunately closed-source - either written by SCO, individual hardware vendors or various other parties. Very few of them can be found on the Internet and the only open source implementations are those in udiref.

@E: Actually it is different than anything else. Just read Robert's quote.