For people interested in implementing file systems

[zse]

I have implemented a small, clean FUSE-based in-memory virtual file system with some neat features, which I think might be a good reference for others, both in terms of how to use the FUSE API as well as the general logical design of UNIX/POSIX file systems.

Have a look if you are interested, I'd be happy about any feedback: https://gitlab.com/z-s-e/vumemfs

I am using the low level FUSE API, as it is in my opinion much cleaner than the high level one, and is a lot closer to how one probably should design an OS filesystem abstraction API.

As a side note, after studying the ext4 filesystem fairly extensively I am interested in designing a similar, but much simplified journaling file system and would like to connect to people who'd enjoy discussing ideas about this. If you know a good place to do that, please let me know.

[CodePoet2005]

hi zse,

looks cool!

If you are looking for a filesystem like ext4, but simpler you could have a look at leanfs.
It has journaling support, but it is not required.

In the past i tried designing a custom filesystem for my bootloader, but i gave up.
Anyway i would be happy to share some ideas on filesystems.
I have implemented some drivers for my tool, so i have some fs experience.

I think this is the correct place to share ideas and ask questions,
if you want to chat, feel free to send me a mail: [email protected]


Cheers,
CodePoet2005

[zse]

Hey CodePoet2005,

Thanks for the suggestion about leanfs, I didn't know about it and it looks somewhat like what I have in mind.

So what is your opinion on leanfs? From a quick look at it, it seems what is called a "block group" in ext* is there a "band". I can see a handful of little things I would have done differently or simplified even further, but one main thing I'd like from the filesystem is a decent block allocation algorithm that prevents fragmentation as much as reasonable (and maybe as a secondary goal, try to keep some locality, similar to what ext4 does https://docs.kernel.org/filesystems/ext ... ators.html).

Given that, I am wondering if one could do better than leanfs (and ext4) by having some more metadata space reserved for the block allocator to store statistics and flags for each block group. I haven't really looked into how modern block allocation algorithms work in more detail, but maybe one idea would be to borrow ideas from memory allocators, in that one reserves some regions for similarly-sized files, and possibly sometimes decide to rather move the already written file blocks instead of fragmenting... thoughts?

[CodePoet2005]

Hi,

Leanfs is in my opinion a nice alternative for ext* filesystems.
The spec is nice and clear and is simpler than ext* filesystems.

Improving lean or ext4 depends on what your specific requirements are. If you are looking for a way to reduce fragmentation a bit, you could have a look at this.

In my filesystem i designed a long time ago, i had the (stupid) idea to have clusters with variable block sizes.

As an example: each cluster would be 256 512-byte physical sectors on disk.
a cluster could have 1024 blocks:
(256 * 512) / 1024 = 128 bytes per block.

A cluster could also have 512 blocks:
(256 * 512) / 512 = 256 bytes per block.

This because my system contained *LOTS* of small config files (less than 512 bytes).
Of course clusters could also have more sectors, just an example.

Basically assuming that your fs is gonna contain fixed sized files, is only useful if this is actually the case, which you dont know (yet?).


As for allocating or searching files and folders, one could keep track of a table (say 32 sectors 32*512=16384) with the most frequently accessed blocks / inodes. on Linux this list would definitely contain your home folder.

Also i have read that windows sometimes defragments your drive as a background task (i dont know if this is true though).
Probably after updates i assume (i hope).

I hope my thoughts can help

Cheers,
CodePoet2005

[zse]

I'm not worried about internal fragmentation, ext4-standard 4KiB fixed block (or "cluster", I don't really use that word and always say "block" for the allocation unit of a fs) size is fine for general purpose, and IMHO fringe use cases with extreme amount of tiny files are better served with special purpose filesystems or databases even. It should be good for normal user data (documents, pictures/media etc) of moderate amount.

[CodePoet2005]

Hi,

I'm not worried about internal fragmentation, ext4-standard 4KiB fixed block (or "cluster", I don't really use that word and always say "block" for the allocation unit of a fs) size is fine for general purpose, and IMHO fringe use cases with extreme amount of tiny files are better served with special purpose filesystems or databases even. It should be good for normal user data (documents, pictures/media etc) of moderate amount.

In this case i dont know if one could make anything "better" then ext4 or leanfs.
Both are already good choices for a decent filesystem, leanfs being simpler than ext4.


Cheers,
CodePoet2005