Mass storage drivers

[berkeleyrc]

So far I've been loading files as GRUB "modules" into memory and using a fake ramdisk driver to read them as filesystems. This has become a problem for larger filesystems like FAT32 where older computers don't necessarily appreciate loading modules that are a few hundred megabytes in size... I'm wondering if I could get suggestions on where to start in terms of writing mass storage drivers. I'd like to be able to read whatever media I'm booting from, and at the moment that seems to be CD-ROMs, but of course I'd like to move to hard disk.

What I'd really like to know is how difficult is it to implement CD-ROM support. Are there other mass storage devices that are easier?

And more on the theoretical side, where is the best place to cache results from the disk? And how much is a good idea to cache? Is it done by the driver that does the reading, or perhaps at some other point in the filesystem (or both)? I guess this could be a matter of taste but I would like to hear other perspectives on the issue.

One last question: how do your storage drivers interface with your filesystem? My ramdisk driver is pretty dumb: it just has a read and write function that pass a length, position and pointer to a buffer. Is this generally sufficient for minimal storage implementations or is life not so simple with real hardware?

[NickJohnson]

For CDs, you will want to write an ATAPI driver; an ATA driver shares some common code with an ATAPI driver, and allows you to use standard hard drives, so you will probably want to write one of those too. Neither should be too tricky: they're both well documented on the wiki.

One last question: how do your storage drivers interface with your filesystem? My ramdisk driver is pretty dumb: it just has a read and write function that pass a length, position and pointer to a buffer. Is this generally sufficient for minimal storage implementations or is life not so simple with real hardware?

Just read/write should be adequate for now. The only things that you might want to add later are a mmap-like interface and something to fine-tune the cache (which could be done with an ioctl-like interface.) Both of those are just optimizations: read/write is all you need for functionality.

[bluemoon]

So far I've been loading files as GRUB "modules" into memory and using a fake ramdisk driver to read them as filesystems. This has become a problem for larger filesystems like FAT32 where older computers don't necessarily appreciate loading modules that are a few hundred megabytes in size...

Woah you have few hundred megabytes of modules to load?
Just to make sure it's not a mistake in the linker script that bloated the image to megabyte scale.
Even you have megabytes of modules installed, I bet you don't really want to load them all at once, try load them on-demand.

What I'd really like to know is how difficult is it to implement CD-ROM support. Are there other mass storage devices that are easier?

I presume you are working on protected mode (on real mode that's easy with BIOS support)
ATAPI cdrom and ATA harddisk are well documented, the osdev wiki should get you started.

And more on the theoretical side, where is the best place to cache results from the disk? And how much is a good idea to cache?

The best place for cache may not necessary be the easiest place.
On modern OS the cache usually work tight with memory manager, everything R/W to any disk are hold at RAM at much as possible while keeping balance with application usage, and until the MM run out of physical memory and decided to purge/page out things.

The caching entity may be logical node can that represent directory, file, remote storage, or system-generated item (/proc or Control-Panel items)
To enable such caching facility you will need an VFS layer on top of every different fs.

Is it done by the driver that does the reading, or perhaps at some other point in the filesystem (or both)?

Little can be done in the driver side, since the driver don't know how many data, and when, you want to read/write, it's hard to do read-ahead or write behind things here.

One last question: how do your storage drivers interface with your filesystem?

storage drivers deal with the IO and access protocol (eg DMA or usb), it r/w raw sectors and have no idea of filesystem layout.
On the other hand the file system module handle the layout of the volume, and call the disk driver to perform the IO operations.

Take your ramdisk as example, they should conceptually be 2 parts:
the "disk driver" - which is just memcpy on sector boundaries.
the "filesystem" - it resolve the filename and know where(the offset) the content stored.

On you have the basic relationship you may expand both disk driver to support more devices, or have more filesystem that store on same media.