BareMetal File System

[IanSeyler]

I'm working on a new native file system for BareMetal OS to replace FAT16. The design is somewhat non-conventional (mainly in regards to all files being contiguous).

The initial specifications: https://github.com/ReturnInfinity/BareM ... 0System.md

Comments/criticism welcomed!

Thanks,
-Ian

[bubach]

I'm just here to push the SFS to gain a bigger user base amongst hobby OS's. Is there any reason not to use it, since your FS seems to be very simple as well? I'd have more understanding for making your own if you needed more functionality. But hey, if you just want to roll your own no matter what - go for it!

[Combuster]

Bubach +1. You design a filesystem with huge blocksizes (= lot of space wasted), without directories (= deliberately hiding needles in haystacks) and contiguous files, and yet you add a bitmap for free blocks which is a horrible structure if you want to efficiently fit a file somewhere.

SFS suits your needs without being braindead.

[VolTeK]

Bubach +1. You design a filesystem with huge blocksizes (= lot of space wasted), without directories (= deliberately hiding needles in haystacks) and contiguous files, and yet you add a bitmap for free blocks which is a horrible structure if you want to efficiently fit a file somewhere.

Correct. I nearly made the same mistake of designing a modified version of the FAT and called it FATTeK. All it really was, was a modified Root directory, and Larger FAT entries. I thought though,

-Why do this, its just another fat that wont be used by anyone else but me, saying that means the only disks formatted by it, will be of my own.

-Why spend time working on this, when there are already designed systems that may do better and already have documentation, and others are using it as well.

I had then stopped and got on with my FAT12/16 drivers.. Almost done too!

My opinion is, unless you haven't already, why not add support for FAT32 instead? It would save a lot of trouble.

[f2]

Designing its own file system is still better than writing drivers for FAT or ext2. Well, I have read the specifications, BareMetal FS is an "extremely simple file system". Indeed, it is simple, and that's good. But I don't agree with some points:
- 2 MiB per block, which is too huge. There's indeed a waste of space, especially with small files.
- The directory only have 64 entries, this is really absurd. By example, If I create 64 small files (< 1 MiB) on a 2 TiB disk, remaining space is unusable. And without support for directories.
- A bitmap scheme is simple and avoid files fragmentation on the disk, but IMHO this is not a good idea. And there is also one inconvenient, when you want to create a new file, the size of this file should be known before.

I have also designed my own file system for my OS. It is also very simple, without making it very limited in features. Here are the differences compared to what I've noticed above:
- 4096 bytes per block, which is enough. For smaller disks, the size of a block is 512 bytes.
- At the beginning of the disk, I have a big index where I store all files records. The number of records depends of the size of the disk. 2048 is the minimum, 1048576 is the maximum. And it supports directories.
- I don't use bitmap scheme, but an array of clusters like on FAT file systems.

Anyway, I encourage you to continue working on your file system.

f2

[VolTeK]

Designing its own file system is still better than writing drivers for FAT or ext2.

That's nice, now state your reason.

[linuxfood]

There appears to be a bug in your specification..

Directory Record structure:

Filename (32 bytes) - Null-terminated ASCII string
Starting Block number (64-bit unsigned int)
Blocks reserved (64-bit unsigned int)
File size (64-bit unsigned int)
Unused (8 bytes)

The create function accepts two parameters:

Name = A null-terminated string no more that 63 characters
Reserved = The number of blocks to reserve for the file

Encoding issues[1] aside, it looks like you'll either crash or silently truncate my filename if I give you a filename >32 bytes long.

Other than that, I'll echo the sentiments of other people here. It looks like you're reinventing the wheel just to do it.
That's not necessarily bad (especially because your goals are different from the goals of most people here), but, this
spec seems to paint you into a corner.

Without suggesting too much, here's a small thought:
Updating that bitmap is going to be fragile. You don't have an obvious or simple way to do a consistency
check on your on-disk structures. Even just adding a header to the start and end of your data blocks could help.
At least then you could scan the whole disk looking for those.

But maybe this filesystem is designed to be the bare minimum necessary to address mostly transitory data and you're
expecting your users will store their important things on some kind of network attached storage?

-B

Edit: Tone, clarity.

[1] Are a byte and a character the same thing in your representation?

[Kevin]

For a read/write file system, requiring contiguous files is wrong. Period.

It may force you to return "no space left" errors even though the vast majority of the disk is free. (Assuming that you don't use complex compaction algorithms which bring your file system into inconsistent intermediate states and are therefore wrong, too)

[turdus]

Agree with the above comments, your fs is wasting space and contains too much limitations.
If you're thinking in continuous spaces allocated for files, and no subdirectories, I suggest to use http://en.wikipedia.org/wiki/Tar_%28file_format%29. I mean .tar, without compression.
Benefits:
- block size matches physical block size (waste as little space as possible)
- can contain special devices, symlinks etc.
- tools for creating the "file system" is already done and available on many systems

[Kevin]

Interesting idea. If someone really insists on having a stupid file system, tar might actually be the best option for its compatibility.

[Rudster816]

Personally, I think FAT32 is the best choice for hobby OS development. I'm not against rolling your own anything when it comes to OS development, because the main purpose of creating your own hobby OS is to learn and have fun, not to have a working piece of software in a timely\efficient manner. That being said, a custom filesystem means that any existing utility you use to make disk images will not work, so you will have to write your own. This will just distract you, and it's another thing that could be buggy. Creating your own filesystem just to make one simple to implement is solving a problem you're creating. There are plenty of very simple filesytems that already exist with a well defined specification for you to follow.

[childOfTechno]

That's not a filesystem, it's a partition table Since all files are contiguous, you should be able to implement a blisteringly fast mmap()

Personally, I think FAT32 is the best choice for hobby OS development.

Except for the complete lack of meta data ...

[brain]

You can't get much better than fat for compatibility, just about every memory stick, floppy, zipdisk, or usb hdd will most likely be fat format no matter what major or hobby os its used in...

Edit: not to mention sd cards and consumer devices like digital photo farms or cellphones...

[Kevin]

the main purpose of creating your own hobby OS is to learn and have fun, not to have a working piece of software in a timely\efficient manner.

The problem is that you can't really design a good file system (or most other things) without knowing existing solutions and their weaknesses and strengths. Of course, you can always learn the hard way how it doesn't work - and this is usually a very effective way of learning, just not really efficient. With file systems, I suppose, it's particularly painful.

[davidv1992]

For the purpose of high performance computing the huge cluster size doesn't matter that much, as small files won't be common anyway. However, there are serious issue points in the rest of your design. Under the assumption that this is meant for HPC, You will wan't some other way of identifying the file in system calls that repeatedly using its name. Name matching will be relatively slow because one needs to repeatedly walk over the table of available files to find which one is requested. Furthermore you are missing the capability of partially reading/writing a file. When consuming/producing big files you will want this, otherwise working with files will consume too much memory.

I would also seriously reconsider using a bitmap to indicate free space. Given the fact that you will be having very few files (64 at most) using a list type datastructure will allow far faster lookup in a far smaller ammount of memory. This could probably be fitted inside 1024 bytes as it will have at most 65 entries.