OSDev.org

Hello everyone,

Suppose i had a UFS-like filesystem (The inode contains direct and indirect pointers to data blocks and file specific informations like access rights, creation time, owner ... etc) and i wanted to add a functionality to support the fragmentation of the last block. I mean, if the file doesn't fill the last block (Uses only a fraction of it - Only some few bytes are written at the end) then instead of allocating a whole block for it, we only allocate a fragment. The system should not keep a pre-allocated pool of free fragments but whenever a file asks for a fragment we can take a free block and divide it into fragments and use one of them. Of course, other fragments from the same block could be assigned to other files. The other fact is that the number of fragments per block is configurable at the moment of installing the filesystem (The fragment has a fixed size). The problem is that i can't figure out what is the best change or structure that i should give to the inode to support such a functionality ? So what do you propose for that ?

As an obvious solution we can have a distinct array of pointers to fragments on the inode (Because the file can use up to N - 1 fragments with N the number of fragments per block) but the size of the array in this case should also be configurable (that means its size changes) but the array is located on the inode and no one wants to have a variable sized inode for it's filesystem, that will be very confusing and will complicate the implementation of the filesystem. That's why i was thinking of something different but i can't figure it out.

If someone is ever wondering, the question is actually from the book : The design of the UNIX operating system by Maurice J. Bach. I am self taught, by the way.

Hi,

I am quite fond of file systems, so I will comment.

First, I don't think this is a good idea. Here are a few reasons:

1) If multiple files possess a single block and that block is later found to be corrupt, you have lost not only one file, but multiple files.

2) What if a file possesses the first part of the fragment block and later needs to be added to. Do you now allocate another fragment block? Soon you will have many fragments. Not good. If you "free" that part of the fragment block to allocate another full block or another fragment block, how do you know that that part is free for use? Now you have to have a whole new file system just to keep track of the fragments.

A file system is (should be) chosen for the type and size of media that will be used. If you know that a specific media device will have many files that will not occupy all of a block, i.e.: wasted space, you choose a file system that accounts for this. If you know that a media device is large enough and you know that the count of files will be minimal, you choose a file system that will store the files so to keep the media access time small, allowing the wasted space because media access time is more important.

Anyway, just my few comments. We all have our opinions. Mine just happens to be against storing multiple files (or fragments) in the same block of the media.

Thanks,
Ben
- http://www.fysnet.net/osdesign_book_series.htm

Hi,

afr0ck wrote:Suppose i had a UFS-like filesystem (The inode contains direct and indirect pointers to data blocks and file specific informations like access rights, creation time, owner ... etc) and i wanted to add a functionality to support the fragmentation of the last block. I mean, if the file doesn't fill the last block (Uses only a fraction of it - Only some few bytes are written at the end) then instead of allocating a whole block for it, we only allocate a fragment. The system should not keep a pre-allocated pool of free fragments but whenever a file asks for a fragment we can take a free block and divide it into fragments and use one of them. Of course, other fragments from the same block could be assigned to other files. The other fact is that the number of fragments per block is configurable at the moment of installing the filesystem (The fragment has a fixed size). The problem is that i can't figure out what is the best change or structure that i should give to the inode to support such a functionality ? So what do you propose for that ?

Assume you have a file system on an older hard disk and you copy the entire partition to a newer SSD using something like "dd" to copy raw data from "/dev/hda1" to "/dev/hdb2". This should work (you should be able to copy file systems like this).

Next assume that the file system was on an older hard disk with 512-byte sectors and the file system originally used a 512-byte block size; but you copied it (using something like "dd") to a newer SSD that happens to have a 4096-byte block size. This should work too (you should be able to copy file systems like this, even if the physical sector size is different), it just might not be 100% optimal for performance. Of course in this case you'd hope that the file system code would allocate "groups of 8 blocks" for new files to improve performance.

Essentially; there's a logical block size (that the file system uses) and a physical block size (that the underlying storage device provides), and these can be quite different. The physical block size is nothing more than an optimisation hint; and (if you really wanted to) you could choose any logical block size you want; including using 3-byte blocks, or 100-byte blocks, or anything else; and it shouldn't effect anything except performance.

Now; what is the real difference between "8192-byte blocks where file system may be split a block into 64-byte fragments" and "64-byte blocks where file system might allocate groups of 128 blocks for performance reasons"? In both cases the file system would be able to allocate individual 64 byte pieces - the only real difference is that the first has to have extra code to manage fragments while the second only manages blocks and doesn't have any extra code that shouldn't already be there.

In other words; why bother with fragments when you can just reduce the block size instead?


Cheers,

Brendan