Implementing your own Filesystem

[Warrior]

Hello,
I am looking to implement my own filesystem so my OS has something standard to work with files. I plan to add support for other filesystems so there won't be a compatability problem between other operating systems and sharing data.

Now, my problem is that I have not found any documents online explaining what to take into account when begging to write one or even what ideas to go with?

Can anyone please suggest good features and some
logic behind how they work please?

Thanks,
Nelson

[Pype.Clicker]

Well, i'll try to sketch out the current "state-of-the-art" in Filesystem design. Note that, even if few people write tutorials about RollingYourOwnFileSystem, there are plenty of docs on existing FS that you might wish to check.

So.

directory hierarchy:
each directory should have a unique path from the "root" to itself, for sanity and checking purpose. virtual hierarchies could be built on top of that, but always have a single tree or you're in for the need of a garbage collector for your FS.

extents
allow the internal structures of your system to store "sector extents" (e.g. contiguous sector ranging from #### to ####). They'll lead to leaner and easier to check sectors list.

inodes
Each file is associated with a small collection of sectors that contains the information about the file such as access timestamps, ownership, size, etc. Make sure you can identify _all_ the blocks of the file from the content of that inode too (even if indirection is used).

room for convenient access rights
i'd advocate for a simple pointer to a "security policy" in each inode rather than having a full list of owner, group, access right, etc. It's rather rare that a single file need specific policy and it's very common that you want to change the policy of several file in a coherent way. Moreover, if all your "user files" carry a single "security ID", once you have parsed it for a given user, you can cache the result of the policy.

think of cache performance first
Surprisingly enough, great FSes are not those that better organize things on the disks: they are those that allow better organization of things in memory. Each chunk of data can be associated a "temperature" that tells how important it is to have it cached in memory. Make sure all your "hot" chunks are in few blocks and that you don't mix things of different "temperatures" in the same block or you'll be loading/saving data you don't need to everytime.

allow symbolic links
Not having symbolic links is a nightmare for the developer. Think of the development of a complex, multi-platform project without the ability of re-linking "arch" symbolic directory to "i386-arch" or "i586-arch" or "alpha-arch" ... gasp.

[DruG5t0r3]

Although I do encourage creating new file systems, I'd say don't even go there. There is plenty of already nice file systems out there (and some that aren't) that you can use. I'd suggest using an open one such as ext2-3. Why add to the fuss of like hundreds of filesystem that we all don't even know about (geez I wish no one ever heard of FAT)

Anyway....back to the point, use an already existant file system its easier and can be read by other OSes.

[Warrior]

The way I see it is I like to do everything myself that way I can add any enhancements I want to the filesystem withought breaking compatability with other OSes because it doesn't exist, and like I said additional support for other filesystems will be there already.

[AR]

Another problem comes from having to write tools to manipulate your disk images, since obviously none of the existing tools will be able to manipulate it.

Do you really want to make a full filesystem or just a Q&DFS?

[Warrior]

Well I did plan in the future (since I'm such an awesome super guy) to implement something lke how MacOSX Tiger has SpotLight to search through files and wanted to have an FS which had room for enhancement withought breaking compatability with anything else, even if it meant writing tools to manipulate it.

Also isn't there a way to add an additional layer under the filesystems to translate all the filesystem operations into a standard API, something like a global filesystem translation system which can unite any read and write to the harddrive or floppy under one global API?

I think something like this would work well.

[Crazed123]

Well, the best way to do that would be to make sure that all read/write devices present a fairly uniform interface to the filesystem driver trying who use them. In other words, it would be an issue of device driver design.

[Pype.Clicker]

Well I did plan in the future (since I'm such an awesome super guy) to implement something lke how MacOSX Tiger has SpotLight to search through files and wanted to have an FS which had room for enhancement withought breaking compatability with anything else, even if it meant writing tools to manipulate it.

Welcome aboard, dude. That "spotlight" feature has amazed me equally.

I have a couple of ideas to share about such higher-level FS features if you like. Also, candy and i are trying to sketch the basis of StarFS design.

[Warrior]

I seem to be unable to reach the page !

[Pype.Clicker]

I seem to be unable to reach the page !

Aaah. One day, sourceforge will be able to handle the load it faces.
Try out now and then, it will eventually work. Sorry for the inconvenience.