OSDev.org

pat wrote:The tutorial mentions in its initial design it only supported 64 files in the root directory, and then goes to mention that it now supports sub-directories, but it didn't say anything about file limits. Does the 64 file per directory limit still exist?

BenLunt wrote:As some of you know, I am quite fond of File Systems, and like to look over almost all I come across. However, my interest isn't the code that is used to access it, but the actual file system content on the target media. i.e.: The way it is stored on the disk, the boot sector, super block(s), FAT(s), bitmap(s), etc.

BenLunt wrote:I see that the link you provide, as well as the links within that page show how to write code to call an existing library, but do you have any documentation on the actual format of the media content of the file system?

Is the documentation at https://github.com/ReturnInfinity/BMFS/ ... 0System.md current, i.e.: to what you are discussing, or is this the original BMFS and you have forked off of it?

BenLunt wrote:This is, as you state, a very simple file system. Please don't think I am disrespecting you or your work, but please let me make a few comments.
(I haven't looked over it in detail, but have glanced and found a few things)

1) Start of files should be indexed by blocks, not bytes. Even though 64-bits allows for quite large volumes, even for byte indexes, start indexes for anything should be block indexes, not byte indexes (indices ?).
2) Is a file limited to 2Megs in size or can it occupy multiple blocks? If so, how do you indicate where these blocks are?
3) 192d (0xC0) bytes for a file name is somewhat excessive unless it includes the path as well.
4) Speaking of, are paths assumed through the directory path, or are paths stored within the file name area (#3 above).
5) How are path names delimited? Forward slash, back slash, space?

Again, don't think I am dis'ing your stuff here. I am just a stickler for details.

Korona wrote:Thank you. I think the disclaimer is quite good and could serve as an example for similar pages (that currently lack a disclaimer).

Korona wrote:]Regarding the page content: The library design seems to be sound (at least for a library that is meant to get started).

Korona wrote:The only thing that I spotted when skimming over it is that the seek() -> read() approach cannot be made thread safe, so it might be wiser to just give a position argument to read().

Korona wrote:Also: Is writing supported?

Korona wrote:Overall, I like the idea of providing a library and host OS tooling to implement a basic file system that can easily be integrated into OSs.

tay10r (from reply) wrote:...what you would want to see in the next release.

• Super block (your "header" structure) that is backed up some where. For example, if the primary header (super block) is at LBA 1024, a backup (secondary) header is at: LBA = ((block count / 2) + 1)
• Bitmap(s) of allocated blocks.
• unlimited file name lengths, but only using what space is needed. i.e.: No wasted space. (i.e.: currently if the file name is "a.img", 0xC0 - strlen("a.img") is a lot of wasted space).
  For example, a file system I was/is a part of is the LeanFS, and it uses a count of 16-byte blocks for filenames, wasting at most 15 bytes of space per filename, yet allowing a file name to be quite long (four digits if I remember correctly).
  Another file system I use, and created myself for numerous reasons, is the FYSFS, and allows unlimited file name sizes, though can waste around 80 bytes of space.
  (Side note: FYSFS was my first file system project, other than the (in)famous FAT file system. I wanted to create something that I could use to test my kernel with to make sure it was completely FS independent, as well as the enjoyment of creating a file system. It served its purpose and now I use the LeanFS file system as my default choice.)
• Streams: The LeanFS uses Extents to indicate how many blocks are allocated for a specific part of the file's data.

  Code: Select all

  struct STREAM {
    bit64u start_lba;  // starting LBA relative to some point: start of volume, start of band, start of ????
    bit32u count;      // count of consecutive blocks used for this stream
  };

• Bands: The LeanFS uses bands, or sections of the media to store all data needed for that part of the file system. For example, the Inode, Metadata, file name, *and* allocation bitmap for any specific file is within a section of the media. This way, if for some reason a portion of the media is destroyed, any remaining bands (sections) are still intact and can be extracted 100%.
• Symbolic links: Allows for numerous "Directory entries" to point to the same file. This has a huge advantage in common used code bases.
  For example, let's say that you have a standard library (source) file that is linked to multiple projects. Rather than having to use the same, some times long path for each project, you can simply create a Symbolic Link directory entry right in the project's main path and use it as a link to the actual library (source) file.
  There are draw backs to this though. What if you move the original file and your file system doesn't have the capabilities to update all links?
• Meta Data: You have already implemented some of this with your directory entry, but what about non-file system specific meta data items.
  For example, the LeanFS allows data to be stored in a file's Inode or a separate Inode linked to this Inode to store non-file system specific meta data, which allows the OS to store information about the app, user, network, etc. currently using the file.

• Variable boot sector space: The FAT file system allows a boot sector code block to use as much disk space as needed (though not all implementations account for this). The Linux Ext 2/3/4 file systems do not. Two blocks. Period. Ouch. Give a variable spaced boot sector allowing a single block or <exaggeration> half the drive if needed</exaggeration>.
• Store the last state of the volume in the super block (header). For example, was the volume previously successfully dismounted? If not, don't try to mount it without first trying to fix it.
• Use magic numbers. i.e.: Give specific, static, values to certain structures of the file system. For example, if the primary super block is ever lost, you can search the disk somewhere in the range of ((block size / 2) - 16) -> ((block size / 2) + 16) for a block that contains certain "magic" numbers to see if there is a valid back up super block. This also allows your driver to verify that you actually loaded a valid Inode before modifying the data it may or may not point to.

BenLunt wrote:

• Super block (your "header" structure) that is backed up some where. For example, if the primary header (super block) is at LBA 1024, a backup (secondary) header is at: LBA = ((block count / 2) + 1)

BenLunt wrote: [*]Bitmap(s) of allocated blocks.

BenLunt wrote: [*]unlimited file name lengths, but only using what space is needed. i.e.: No wasted space. (i.e.: currently if the file name is "a.img", 0xC0 - strlen("a.img") is a lot of wasted space).
For example, a file system I was/is a part of is the LeanFS, and it uses a count of 16-byte blocks for filenames, wasting at most 15 bytes of space per filename, yet allowing a file name to be quite long (four digits if I remember correctly).
Another file system I use, and created myself for numerous reasons, is the FYSFS, and allows unlimited file name sizes, though can waste around 80 bytes of space.

BenLunt wrote: (Side note: FYSFS was my first file system project, other than the (in)famous FAT file system. I wanted to create something that I could use to test my kernel with to make sure it was completely FS independent, as well as the enjoyment of creating a file system. It served its purpose and now I use the LeanFS file system as my default choice.)

BenLunt wrote:
[*]Streams: The LeanFS uses Extents to indicate how many blocks are allocated for a specific part of the file's data.

Code: Select all

struct STREAM {
  bit64u start_lba;  // starting LBA relative to some point: start of volume, start of band, start of ????
  bit32u count;      // count of consecutive blocks used for this stream
};

BenLunt wrote: [*]Bands: The LeanFS uses bands, or sections of the media to store all data needed for that part of the file system. For example, the Inode, Metadata, file name, *and* allocation bitmap for any specific file is within a section of the media. This way, if for some reason a portion of the media is destroyed, any remaining bands (sections) are still intact and can be extracted 100%.

BenLunt wrote: [*]Symbolic links: Allows for numerous "Directory entries" to point to the same file. This has a huge advantage in common used code bases.
For example, let's say that you have a standard library (source) file that is linked to multiple projects. Rather than having to use the same, some times long path for each project, you can simply create a Symbolic Link directory entry right in the project's main path and use it as a link to the actual library (source) file.
There are draw backs to this though. What if you move the original file and your file system doesn't have the capabilities to update all links?

BenLunt wrote: [*]Meta Data: You have already implemented some of this with your directory entry, but what about non-file system specific meta data items.
For example, the LeanFS allows data to be stored in a file's Inode or a separate Inode linked to this Inode to store non-file system specific meta data, which allows the OS to store information about the app, user, network, etc. currently using the file.[/list]

BenLunt wrote: As for a Journal and encryption, these are good things to have for a file system, but be careful, both of these items will make a simple file system, instantly not so simple anymore.

BenLunt wrote: There are a few other things I want to list:

• Variable boot sector space: The FAT file system allows a boot sector code block to use as much disk space as needed (though not all implementations account for this). The Linux Ext 2/3/4 file systems do not. Two blocks. Period. Ouch. Give a variable spaced boot sector allowing a single block or <exaggeration> half the drive if needed</exaggeration>.
• Store the last state of the volume in the super block (header). For example, was the volume previously successfully dismounted? If not, don't try to mount it without first trying to fix it.
• Use magic numbers. i.e.: Give specific, static, values to certain structures of the file system. For example, if the primary super block is ever lost, you can search the disk somewhere in the range of ((block size / 2) - 16) -> ((block size / 2) + 16) for a block that contains certain "magic" numbers to see if there is a valid back up super block. This also allows your driver to verify that you actually loaded a valid Inode before modifying the data it may or may not point to.

+= offset