"designing generic storage device driver interface"

[GLneo]

This was brought up in another thread but I got interested making one, but how, any suggestions?

[Brendan]

Hi,

This was brought up in another thread but I got interested making one, but how, any suggestions?

I'd first decide whether storage device drivers can operate on whole blocks only (e.g. read 3 sectors starting at sector 4) or if they should support smaller transfers (e.g. read 3 bytes starting at byte 4). Then, if they operate on whole blocks only decide if the size of the block can be different for each device (e.g. 512 bytes for hard drives and 2048 bytes for CD-ROM), or if blocks are fixed size (e.g. 4096 bytes regardless of what sort of device it is).

Once this is done I'd write a list of operations it needs to support. For example:

• - read data
  - write data
  - open device
  - close device
  - seek (?)
  - flush (?)
  - format device (?)
  - get block size (?)
  - get device information (?)

Then for each operation determine how it should work. Can the device have an optional cache (which may need to be flushed), or is caching done at higher levels only (e.g. by the file system code)? Are data transfers done from the current position or from any position - e.g. "read(size)" and "seek(position)", or just "read(size, position)".

Then decide how removeable media will be handled - if the device driver knows a CD was ejected, does it send a "media changed" notification to it's client, or does it return a "media changed" error the next time the client tries to read or write?

Does the "client" block until an operation is complete or can it continue doing other things while it waits? Should storage device drivers be capable of handling a list of operations, or are operations always done one at a time? Can the storage device driver queue operations and perform them in any order to minimize seek times? Can different operations have different "priorities", so that swap space data that's needed urgently is transfered before Mr User's recipe for apple pie (regardless of the order the operations were requested)? Can an operation be cancelled if it was queued but not started yet? How is status and errors returned?

How does file system code "mount" a device? Can the device be mounted in different modes (read-write, read-only, etc), and if it can, can it handle several clients (e.g. can 3 different clients use it at the same time in "read-only" mode)?

What about disk partitions? Is the hard disk driver mounted by a partition manager which is mounted by file systems? Does a hard disk driver have to handle the partitions itself?

How does other software know the device is present? Is there a special function to find out if it's there or not, is there a special directory (like the "/dev/" directory on Unix), or perhaps the OS's IPC mechanism takes care of it somehow (named pipes for example)?


Before deciding any of the above "on paper", work it out roughly in your head, then ask yourself if it should be combined with the normal file I/O interface. For example, when file system code mounts a device does it use the normal "fopen()" function, and then use the normal "fread()" and "fwrite()" functions until it's unmounted (with the "fclose()" function)?

If you leave these interfaces seperate, then you'll also need to write a seperate "file I/O interface" specification.

If you do choose to combine the storage device interface with the normal file I/O interface, then you'll need to design both at the same time and work out some unusual things (like what happens when someone tries to append data to a storage device, or tries to get device information from a normal file). In this case, it might also be a good time to consider the interface for other types of devices (video, keyboard, ethernet, printers, etc). For example, will the user be able to do "cp /dev/scn0 /dev/lpt0" to copy data from the scanner to the printer? Then there's archives - implementing them as file systems and supporting things like "cp myFile archive.zip/myStuff" could be quite useful.

Anyway, lots to think about...


Cheers,

Brendan

[Solar]

You might want to check out http://www.projectudi.org. They did just the thing.

[Brendan]

Hi,

You might want to check out http://www.projectudi.org. They did just the thing.

IMHO UDI is like a jockey sitting ontop of a horse's corpse, wipping away trying to go faster while maggots are cleaning the bones.

To prove how portable it is they did partial implementations on top of several different versions of UNIX, but none of those UNIX projects actually adopted it. From what I can tell it'd be a complete nightmare to port to a micro-kernel (or possibly anything that isn't UNIX).

The UDI web site contains the latest version of the specification (version 1.01, the second version, released 5 years ago or around 18 months after version 1.00 was released), the "Products and Source Code" page contains more dead links than I've seen in a long time, and the SourceForge project seems to have been mostly inactive since 2001.

Despite all of this, if you're writing yet another UNIX clone then, um, start with a Linux or BSD kernel and get drivers that actually work instead.

Anyway, I'm going to design a single diesel engine that can be used for portable generators, motorbikes, cars, trucks, helicopters and jumbo jets. It'll revolutionize the industry and make it much easier for spare parts suppliers and manufacturers - stay tuned...


Cheers,

Brendan

[viral]

Hello.....
I dont know if this is the place to ask this question. But it is linked with device driver so i dint created new thread.
I am trying to use "C++ Virtual Functions" for device driver code. Am having a "class BlockDevice":
class BlockDevice
{
virtual int read(char *buffer, int block_id, int no_of_blocks);

virtual int write(char *buffer, int block_id, int no_of_blocks);
};

This class is used as base class for class FloppyDriver:

class FloppyDriver : public BlockDevice
{
int read(char *buffer, int block_id, int no_of_blocks);
int write(char *buffer, int block_id, int no_of_blocks);

};


Here i use the virtual function concept to call read or write call from floppy driver:

BlockDevice *blkdev;
FloppyDriver fdc;
blkdec = &fdc;

blkdev->read( ... );

Here this code works when fdc is declared locally (above blkdec = &fdc), but kernel crashes with page fault when i declare fdc globally.....

Is it so that virtual functions wont work here as they need runtime support??

[bluecode]

@viral: C++ virtual functions don't need runtime support. Just for pure virtual functions you need an extra function (function name depends on the compiler, everything is explained in the OSFAQ). But for static/global Objects you need extra support, which is also described in the OSFAQ. That's because the constructor of these objects has to be called before main().

[kataklinger]

Templates also don't need runtime support (it's great for linked lists and other data structures)

[Pype.Clicker]

I'd first decide whether storage device drivers can operate on whole blocks only (e.g. read 3 sectors starting at sector 4) or if they should support smaller transfers (e.g. read 3 bytes starting at byte 4). Then, if they operate on whole blocks only decide if the size of the block can be different for each device (e.g. 512 bytes for hard drives and 2048 bytes for CD-ROM), or if blocks are fixed size (e.g. 4096 bytes regardless of what sort of device it is).

Considering the amount of questions brendan came with, i'd be tempted to take the day off to think about it all. First of all, try to get a clean cut between what's being handled and what's not. Block caching, for instance, is not device-specific: if a caching policy is good for e.g. ATA disks, it is probably equally good for SCSI disks. Similarily, E2FS and NTFS will equally don't want to bother with caching: all they need is the ability to say "i need the disk to be sync'd before further operation go on" or "here is a barrier: write these stuff first before anything newer is written". Journalled FS is also likely to need unbuffered writes at some times (for the journal), so i'd dare to say caching should probably be a 'companion' service that you can use between FS and block device, but not a part of either FS or block device.

Once this is done I'd write a list of operations it needs to support. For example: read data, write data, open device, close device, seek, flush, format device, get block size, get device information

you'll certainly need a comfortable way to access all the information about a given device - not only the block devices, but also USB devices, human-interacting devices, display devices, etc. Thinking of how you could perform queries on those information, how they could be reported, etc. should imho receive great care if you want to avoid the nightmare of /proc (which is a stockpile of info for the programmer, but a horror to interface with at kernel level, imvho)

I'd also advocate for a simple, stateless interface at device level, that is, transfer blocks only, and each request identifies which block(s) is to be transferred. The reason for doing so is that it will ease the handling of requests coming from different 'sources' (e.g. multithreaded access to the FS with each client accessing a different file shouldn't require multiple connections between the FS entity and the device driver).

Then decide how removeable media will be handled - if the device driver knows a CD was ejected, does it send a "media changed" notification to it's client, or does it return a "media changed" error the next time the client tries to read or write?

Once again, this can be hopefully enough designed as a 'companion' for the core driver.

[GLneo]

IMHO UDI is like a jockey sitting ontop of a horse's corpse, wipping away trying to go faster while maggots are cleaning the bones.

WOW!!!

do all unix clones use the same system???

[Brendan]

Hi,

IMHO UDI is like a jockey sitting ontop of a horse's corpse, wipping away trying to go faster while maggots are cleaning the bones.

WOW!!!

do all unix clones use the same system???

AFAIK each different UNIX clone uses it's own unique driver system with it's own unique drivers. This is actually quite silly when the systems are so similar - UDI could have been a good thing for these UNIX clones.

If UDI actually stood for "UNIX Driver Interface" then it may have become extremely useful. Consider my "Uniform Diesel Engine" - if this engine was designed for trucks only, then it'd be a very good idea, but trying to design an engine that suits everything is impossible (guaranteed to fail).


Cheers,

Brendan