Yo:
I don't normally press issues and pursue an argument like this, but I think it would be very meaningful to prove that UDI does cater to all of these needs quite ably, mostly for the benefit of other readers who might be interested in adopting it. This post is terribly long, but it needed to be this long because Brendan raised a lot of points, each of which needed a decent amount of expansion.
Brendan wrote:If you look at UDI, you'll find that the specifications themselves only cover some stuff (e.g. disk drives and network cards) and doesn't cover other stuff (e.g. 2D video, 3D video, GPGPU, sound input/output, GPS, accelerometers, cameras, printers, scanners, force feedback, etc); and the devices it does cover are relatively easy to support without UDI, and the devices it doesn't cover are hard. Of course things get much worse when you try to find any actual drivers (you will only find drivers for simple and obsolete hardware).
All of these can easily be supported by any individual; this is the fundamental purpose of Metalanguages. They exist to enable new APIs for device-classes to be created and drafted as needed. If you have a non-ubiquitous device that you wish to deploy, you can define your own custom API, deploy it, and submit your draft API for peer review. Over time, the API will be rounded off by the UDI Project and the community of UDI participants, and when the "Standard" UDI API is agreed upon, you can then use that Metalanguage in your new deployments.
In addition, it is not in any way weird for a driver to export multiple APIs. Many graphics drivers export both OpenGL and D3D. This is nothing novel or unique to UDI. As expected of a framework in its infancy, it will have areas you will need to pioneer. If you don't like pioneering, feel free not to use UDI.
Brendan wrote:Worse; if you look at the UDI specs themselves you'll notice they do "minimal functionality" well; but anything beyond that either doesn't exist or is platform dependent
This is a false assertion, as I will demonstrate below. By individually tackling each of these bullet-points.
Brendan wrote:If you want whole disk encryption, you're screwed
A driver that supports full-disk encryption can and will do this regardless of the presence or absence of OS APIs for it. It needs only import the necessary libraries and it will be able to encrypt and decrypt the data. If there is a need to enable the OS to give commands or configuration directives to the driver, in UDI this can be easily and readily supported using the UDI Generic I/O metalanguage.
A custom userspace utility can be created, and this utility can communicate with the driver to control the encryption settings, etc, and it will communicate with the driver over the UDI Generic I/O Metalanguage child-bind channel. A
fully compliant UDI OS doesn't need to know about the APIs that a driver exports or what they do; Programs can use them either way, and OS APIs don't need to be a "middle-man". This isn't unique to UDI either; IOCTLs, etc are similar in nature.
Brendan wrote:If you want your a sane IO priority scheme, you're screwed
Only the kernel has the metadata required to know which processes' requests should be considered high-priority, and usually the I/O scheduler is a feature of the OS. If you wish to do aggressive I/O prioritization, you can fragment requests at the kernel level, and send them to the device in fragmented portions. This will enable you to have a higher chance of applying very fine I/O prioritization to requests. This would be the case whether or not you used UDI, so this is also nothing particularly revealing.
Brendan wrote:If you have special synchronisation requirements (e.g. "notify me if sector XYZ is modified") you're screwed
This point is fair...but only in so much as this is not a commonly traveled path by contemporary kernels. Usually, you do notification at the file-system level, on the file-level abstraction, and not on the sector-level concretion. This is a very odd feature to want at all.
However, should you want to give applications in your userspace notifications when particular hardware sectors are written to, you can do this very easily in UDI anyway:
all requests that go both in
and out of a UDI driver will pass through a UDI Metalanguage Library, and this is unavoidable. You can write a custom Metalanguage Library for your kernel that will intercept all IPC for say, disk-I/O, and implement this notification framework out-of-band, without needing to modify any of the drivers. From the documentation for
udi_mei_driver_error():
UDI Core Specification, Volume 2, Page 174, Section 28.4 wrote:The metalanguage library is not required to check for invalid parameters or other illegal conditions, and should normally not check for invalid parameter values that would (per the metalanguage definition) be expected to be checked in the target driver. Illegal conditions that are detectable only by a portable metalanguage library (not by either the driver or the environment) should be checked for in untrusting metalanguage library implementations.
By implication, Metalanguage Libraries can be used as "interceptors" to do additional, API-specific error checking and security operations, though this is not required. You can then extend this to suit your needs, and provide a UDI Disk Metalanguage library that will intercept disk I/O requests (on the ACK/response side of the asynchronous call) and dispatch messages to the kernel if the sectors involved match those you need notifications for. The kernel can then dispatch notifications to the applications which are listening for such notifications. Note that no driver support or modification of driver code is necessary, and you can add this functionality
even to drivers that are distributed only in binary form. That is, you can do it without needing to have the source code for the disk driver, and do it in a generic, portable way too.
I suspect that Brendan wants this feature for security. Perhaps he wants to monitor for changes to the Volume Boot Sector, etc. At any rate, UDI can more than ably, and portably accomplish what he wants.
Brendan wrote:If you want additional special purpose interfaces (e.g. possibly one designed for emulators, where a disk controller is assigned to a virtual machine such that the guest OS inside the virtual machine has "almost direct hardware access") then you're screwed
This is simply not true, either for UDI, or for any other kernel's disk API. If you want to connect at a low level to a device, you can make use of IOCTLs or that kernel's equivalent. This is not a problem in any modern kernel, tbh, unless I am misunderstanding what you're advocating :O
Brendan wrote:If you want "secure erase" then you're screwed
When a UDI Storage/Disk Metalanguage is drafted, I will be
very astounded if it does not include support for secure erasure, given that this is a very ubiquitous disk driver feature.
Brendan wrote:If a file system needs to
"trim", then you're screwed
I don't know enough about SSDs to comment authoritatively.
Brendan wrote:For any/all power management, (e.g. turning hard disks off when they're idle, disabling pre-fetching when you're running on battery, etc) you're screwed[/list]
Power management is handled by udi_devmgmt_req(), and in combination with ACPI, I don't see why this is impossible. Out-of-band special commands can be sent to configure the driver using a userspace utility if absolutely necessary. This would not be unusual either. Conventionally today, we see many such drivers: an AMD Catalyst Driver for example, comes with a userspace daemon that augments the kernel with support for its less universal features. This is not novel behaviour.
If a kernel wishes to specify custom APIs for hardware vendors to allow it to control certain aspects of devices, it can specify such an API and try to get hardware vendors to adopt it in their drivers. Then, those kernels which are interested in the custom API can establish child-bind channels on that custom Metalanguage and do whatever they need to do with those custom APIs (enable_prefetch_req()/disable_prefetch_req()). The only problem here is
Independent Hardware Vendor adoption of your custom requirement, and the absence of UDI does not improve your prospects here.
Brendan wrote:In addition to all of that; you end up with another layer of bloat. Rather than having drivers that support the OS's native interfaces directly, you end up with a bunch of additional "shims" between the OS's native interface/s and UDI.
If you implement the UDI environment natively, you will gain native support, and thereby native performance. If you implement the UDI environment as a layer on top of another interface, you will gain second-class-citizen performance, since your UDI support is second-rate. This argument is like saying that "you end up with shims between the OS's native interface and WINE when you use WINE to support the NT API."
Brendan wrote:Finally; if you ever actually do find a driver that supports UDI, you still can't assume it will actually work - you have to do your own "quality assurance" testing to be sure (which means finding/purchasing the appropriate hardware/device); and maintain some sort of blacklist/whitelist (especially when you start considering the possibility of "trojan UDI malware"). You also might not be able to (e.g.) bundle the driver with your OS, because the copyright isn't yours and may be "undesirable" for whatever reason (e.g. proprietary, GPL), and/or because the driver may be owned by a known patent troll like SCO.
This is not different from any other software that is distributed; drivers are software, like applications. If you want a "trusted driver" framework, you can set about doing this with relative ease. The only barrier is time.
Brendan wrote:The end result is that you need to do a lot of work just to support UDI, and in return for this large amount of work you get a few irrelevant drivers
The same goes for inventing a custom API. The only difference is that UDI is a very well-thought-through, scalable API, and you will not need to go through design re-iterations with UDI. With your own custom API, due to short-sightedness, the average kernel developer in this community will spend several years butting his head before he eventually, after many re-iterations, settles on something that is adequately well-designed and stable.
All the while, he would have needed to repeatedly update
all of his drivers each time he had to redesign. UDI saves him all of this time. It looks like a lot of time spent up-front on implementing UDI, but this is regained in time saved not having to regressively redesign and update dozens of drivers repeatedly, possibly 15, 20, maybe 30+ times over 2-3 of years, and never having a stable driver API.
--Peace out,
gravaera