SimpleFS - Missing from WIKI and Brendan's site
SimpleFS - Missing from WIKI and Brendan's site
Hi guys and girls,
After strugglling to redesign my os, am back at the stage to implement a file system.
I remembered our SimpleFS discussion and came looking for the specification.
Could not find it in WIKI or on Brendan's site so did a search and found the topic discussion.
I feel we should have a link to this file system in our WIKI as it is a simple and nice one to implement.
Note to Brendan:
You don't have a link to the specification on your Main web site. Could go under Documentation or Specification.
Here is the link to SimpleFS -
http://bcos.hopto.org/sfs.html
well, back to coding.....
After strugglling to redesign my os, am back at the stage to implement a file system.
I remembered our SimpleFS discussion and came looking for the specification.
Could not find it in WIKI or on Brendan's site so did a search and found the topic discussion.
I feel we should have a link to this file system in our WIKI as it is a simple and nice one to implement.
Note to Brendan:
You don't have a link to the specification on your Main web site. Could go under Documentation or Specification.
Here is the link to SimpleFS -
http://bcos.hopto.org/sfs.html
well, back to coding.....
Re:SimpleFS - Missing from WIKI and Brendan's site
Have sfs bein implemented and tested before ?
Re:SimpleFS - Missing from WIKI and Brendan's site
Am going to implement it soon. Will give feedback about the process.
Re:SimpleFS - Missing from WIKI and Brendan's site
Reading this has given me several good ideas. The disk layout already is very similar to my "SensoryFS". ;D I'll post info about that soon, if anyone wants it.
Re:SimpleFS - Missing from WIKI and Brendan's site
Brendan, is that spec. finished?
- Pype.Clicker
- Member
- Posts: 5964
- Joined: Wed Oct 18, 2006 2:31 am
- Location: In a galaxy, far, far away
- Contact:
Re:SimpleFS - Missing from WIKI and Brendan's site
imho, the reason why it's absent from the FAQ is pretty obvious: it's still draft at the moment (so there's no reason to present it at the same level as FAT or BFS or whatever) and brendan has decided to host the specs himself ...
Re:SimpleFS - Missing from WIKI and Brendan's site
I don't know if I'm still on time, but I'd like to propose a change to that draft specificacion. By the way, I'm really looking forward to having it finally implemented!! ^_^
Well, to the proposal. At the end of the "File System Format Overview" section, the draft states:
So I propose a new wording like this:
Oh, and this is quite like the way FAT-32 (really FAT-28) handles that top 4 bits: a compliant driver ignores them both when reading and writing, but writing a new entry will set them to 0. Should we consult the patent office?
PS: I also think that continuation entries should have an ID byte, leaving just 63 for the file name in each. That would make it easier for the driver to build itself a "FS map" and would also allow later expansions of SFS with new file entry types. I realise that 32 entry types (0x00 to 0x1F) should be enough for a "simple" FS, but marking ALL entries with an ID byte looks a lot cleaner to me.
Well, to the proposal. At the end of the "File System Format Overview" section, the draft states:
I understand the purpose of this is to guarantee forward compatibility, but it won't work without destruction! Suppose some "SFS 2.0" finds an use to one of those reserved field - say, for permissions info or whatever else. When an SFS 1.0 driver reads it, it will ignore it - as it should. But what if it then rewrites it, i.e., because of a compaction? The field will be zeroed out. Even the BPB and the boot code should be zeroed out, since they are marked as reserved!All fields marked as "Unused/reserved" must be filled with zeros when written to the media, and ignored when read from the media.
So I propose a new wording like this:
That takes care of 1) legacy structures such as the BPB and future compatibility (because the driver will not zero out zones it does not understand) and 2) current implementation, since, if an entry with reserved fields is written anew, they will have their default value (of zero).All fields marked as "Unused/reserved" must be ignored when read from the media, and kept unaltered when writing to it. The default value for these fields in new entries is all bits zeros unless otherwise stated.
Oh, and this is quite like the way FAT-32 (really FAT-28) handles that top 4 bits: a compliant driver ignores them both when reading and writing, but writing a new entry will set them to 0. Should we consult the patent office?
PS: I also think that continuation entries should have an ID byte, leaving just 63 for the file name in each. That would make it easier for the driver to build itself a "FS map" and would also allow later expansions of SFS with new file entry types. I realise that 32 entry types (0x00 to 0x1F) should be enough for a "simple" FS, but marking ALL entries with an ID byte looks a lot cleaner to me.
Re:SimpleFS - Missing from WIKI and Brendan's site
By the way, where's Bendran's site? I'd like to take a look at the current draft.
Re:SimpleFS - Missing from WIKI and Brendan's site
Oh, sorry, I didn't realise that the address in the first post was Brendan's site... I thought it was just a mirror. Thanks a lot.
Then, the draft hasn't changed since January?
Then, the draft hasn't changed since January?
Re:SimpleFS - Missing from WIKI and Brendan's site
Not as far as I know, thats about the time when it was discussed in the original thread.
Re:SimpleFS - Missing from WIKI and Brendan's site
Hi,
For "SFS 2.0", because it's impossible for SFS 1.0 driver to know about any SFS 2.0 extensions, compaction (or any other changes) done by the "SFS 1.0" code will probably trash any extra "SFS 2.0" data. In this case the best behaviour would be for an SFS 2.0 disk to be converted back to the SFS 1.0 format when it's modified, so that none of the "SFS 2.0 data" is relied upon after that.
IMHO the section on "Super-Block Format" should include a note saying that the SFS 1.0 code should change the "Simple File System version number" field to 0x10 if any data is changed (and if it's not already set to 0x10), and the section called "Index Area Format" should say that any unknown index area entry types should be treated as "unused" and set to the type 0x10 (unused) during compaction or other changes.
This would mean that:
Of course rather than loading the entire index area into memory it could be done by mapping the partition into memory (e.g. memory mapping the file "/dev/fd0"), or as part of a sector cache.
Cheers,
Brendan
For compatibility (reserved areas used for DOS/FAT, partition tables, etc), compliant SFS 1.0 code won't write to these areas (except during formatting, where you'd expect everything to disappear anyway).Habbit wrote:I understand the purpose of this is to guarantee forward compatibility, but it won't work without destruction! Suppose some "SFS 2.0" finds an use to one of those reserved field - say, for permissions info or whatever else. When an SFS 1.0 driver reads it, it will ignore it - as it should. But what if it then rewrites it, i.e., because of a compaction? The field will be zeroed out. Even the BPB and the boot code should be zeroed out, since they are marked as reserved!
For "SFS 2.0", because it's impossible for SFS 1.0 driver to know about any SFS 2.0 extensions, compaction (or any other changes) done by the "SFS 1.0" code will probably trash any extra "SFS 2.0" data. In this case the best behaviour would be for an SFS 2.0 disk to be converted back to the SFS 1.0 format when it's modified, so that none of the "SFS 2.0 data" is relied upon after that.
IMHO the section on "Super-Block Format" should include a note saying that the SFS 1.0 code should change the "Simple File System version number" field to 0x10 if any data is changed (and if it's not already set to 0x10), and the section called "Index Area Format" should say that any unknown index area entry types should be treated as "unused" and set to the type 0x10 (unused) during compaction or other changes.
This would mean that:
- An SFS 1.0 disk can be read by an SFS 2.0 driver, and any writes may convert the disk to SFS 2.0 format (in whole or in part).
An SFS 2.0 disk can be read by an SFS 1.0 driver, but any write will convert the disk to SFS 1.0 format.
As it is, a file system driver can load the entire index area into memory and then treat any directory or file name as a normal string without caring if is uses continuation entries or not (values from 0x00 to 0x1F never represent valid UTF8 characters). The means a file name could be 54321 characters long and stretch across lots of continuation entries, but you can still do something like "printf("%s\n", fileName);"....Habbit wrote:PS: I also think that continuation entries should have an ID byte, leaving just 63 for the file name in each. That would make it easier for the driver to build itself a "FS map" and would also allow later expansions of SFS with new file entry types. I realise that 32 entry types (0x00 to 0x1F) should be enough for a "simple" FS, but marking ALL entries with an ID byte looks a lot cleaner to me.
Of course rather than loading the entire index area into memory it could be done by mapping the partition into memory (e.g. memory mapping the file "/dev/fd0"), or as part of a sector cache.
It will be a relatively long period of time before my OS is capable of supporting any file system - I haven't been working on SFS much because of this...Habbit wrote:Then, the draft hasn't changed since January?
Cheers,
Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Re:SimpleFS - Missing from WIKI and Brendan's site
I agree with that, but what i was saying is that the current draft of the specification does not make that clear: it states that reserved fields must be filled with zeros when written. That was the point of my change proposal.Brendan wrote: For compatibility (reserved areas used for DOS/FAT, partition tables, etc), compliant SFS 1.0 code won't write to these areas (except during formatting, where you'd expect everything to disappear anyway).
Simple solutions for a simple FS:Brendan wrote: For "SFS 2.0", because it's impossible for SFS 1.0 driver to know about any SFS 2.0 extensions, compaction (or any other changes) done by the "SFS 1.0" code will probably trash any extra "SFS 2.0" data. In this case the best behaviour would be for an SFS 2.0 disk to be converted back to the SFS 1.0 format when it's modified, so that none of the "SFS 2.0 data" is relied upon after that.
(...)
This would mean that:
- An SFS 1.0 disk can be read by an SFS 2.0 driver, and any writes may convert the disk to SFS 2.0 format (in whole or in part).
An SFS 2.0 disk can be read by an SFS 1.0 driver, but any write will convert the disk to SFS 1.0 format.
- For a "SFS 2.0" driver, if any 2.0 version gets released, the developer will have access not only to the 2.0 spec, but also to the 1.0. So, when I do [tt]mount -t sfs /dev/fd0[/tt] and the floppy in the drive is an SFS 1.0 disk, the driver should just behave as a 1.0 one and print a diag message like The problem might be with the SFS 1.0 driver, whose developers, because of that d*mned time-space continuum, no-cloning theorem and other stupid things of quantum physics, won't have access to any copy of the (by 2087) widespread SFS 2.0 spec. But then, another simple solution with my poor example of [tt]mount -t sfs /dev/fd0[/tt]. This time with an SFS 2.0 floppy and a 1.0 driver. The drive should be mounted read only and a warning issued like this:
Code: Select all
mount: sfs20: Warning: selected volume is SFS 1.0. Use --force2 to mount it as a SFS 2.0 (new features such as ****** won't be readable by 1.0 drivers)
Code: Select all
mount: sfs: Warning: selected volume SFS version is '2.0' (last supported driver version is 1.0) - mounted as read only. Use --forcerw to mount as read-write (could destroy some unrecognized structures) or reformat the volume as SFS 1.0
And, maybe for the next 2.0 spec, just after the other paragraph:SFS compliant drivers must refuse to mount a volume with a greater version than the last driver supported one as read-write except through a special switch, and issue a warning in either case when mounting such a volume.
And, for the record: SFS is GREAT!! Long live SimpleFS!! ;DAn SFS 2.0 compliant driver should also be backwards compliant with all older SFS versions, and, when mounting a volume with an older version of the filesystem, abide to the correspondant specification unless a newer version is specifically selected. If the newer driver is not compatible with the older version, it must refuse to mount the volume read-write unless overriden by a special switch. In both cases (mounting as an older version or mounting read-only because the older version is not supported), a warning must be issued. A filesystem upgrader utility may also be provided.
- Combuster
- Member
- Posts: 9301
- Joined: Wed Oct 18, 2006 3:45 am
- Libera.chat IRC: [com]buster
- Location: On the balcony, where I can actually keep 1½m distance
- Contact:
Re:SimpleFS - Missing from WIKI and Brendan's site
By reading the specs, the only reason to take on SFS is because its public domain - for the rest its close to FAT32 with all its disadvantages and none of its advantages:
basically theres no support for fragmentation: i.e. unless your file is one block big you can have 50% free space and not be able to store it without having to defragment first.
As for directories, speed is terrible finding one directory's contents - you'll always have to run over the entire index. That and the redundancy in filenames makes it a horror to work with directories
So FAT is in my opinion superior and regarding complexity its not such a step ahead, so i'll probably stick with that. (I'll probably prefer to make an attempt at Reiser or XFS than going for sfs)
Apart from that, SFS is the perfect fs for those wanting to have something easy without worrying about complex algorithms, and not have windoze mess with it. definately a good experiment for newbs on the subject.
And if you get a working implementation, i'll probably promote it as such.
Before that, i dont think it should be in the Wiki
I do however, wonder when the sfs option will appear in the linux kernel config ;D
basically theres no support for fragmentation: i.e. unless your file is one block big you can have 50% free space and not be able to store it without having to defragment first.
As for directories, speed is terrible finding one directory's contents - you'll always have to run over the entire index. That and the redundancy in filenames makes it a horror to work with directories
So FAT is in my opinion superior and regarding complexity its not such a step ahead, so i'll probably stick with that. (I'll probably prefer to make an attempt at Reiser or XFS than going for sfs)
Apart from that, SFS is the perfect fs for those wanting to have something easy without worrying about complex algorithms, and not have windoze mess with it. definately a good experiment for newbs on the subject.
And if you get a working implementation, i'll probably promote it as such.
Before that, i dont think it should be in the Wiki
I do however, wonder when the sfs option will appear in the linux kernel config ;D
Re:SimpleFS - Missing from WIKI and Brendan's site
Well, the point in creating a Simple File System is to KISS, It does not even try to become something like FAT, which could be used for rather large - 2 GiB - hard drives. It is basically thought for small media like floppies and especially small boot partitions where the data does not change for long spans of time.Combuster wrote: By reading the specs, the only reason to take on SFS is because its public domain - for the rest its close to FAT32 with all its disadvantages and none of its advantages:
By the way, it does not have one of FAT32's disadvantages: it is not patented by Microsoft.
There is yet no support for fragmentation in SFS 1.0 draft. You also can collaborate! If you devise an algorithm that allows SFS to accept fragmented files while still remaining Simple, and want to contribute it to the public domain or whatever free license SFS ends with, propose it!Combuster wrote: basically theres no support for fragmentation: i.e. unless your file is one block big you can have 50% free space and not be able to store it without having to defragment first.
On the other side, given the use that SFS is going to have (floppies and boot media), I don't see a really big problem with that. SFS defragmentation (whats the name? compaction?) seems a very simple operation to me.
I also can't see a problem with going through the whole index area - for small media. I do think that the entries are a bit bloated: 64 bytes that could be reduced to 48 by trimming the file start & end block and size from 8 bytes to 4 (Are you really storing a >4GiB DVD image in a 1.38MiB floppy? I totally NEED your compressor!!) and reducing the entry name bytes by 4 (for file entries; in others all 16 could be taken from "name" or "unused"). But even with this bloat, an index area with 256 entries (let's say the source of your microkernel) would be roughly 16KiB, or about 2% of a 1.38MiB floppy. Not too much for a filesystem easy as hell to parse: you can even keep the whole index in memory. And remember that this index area can be expanded (and, I would propose, shrinked were it needed and possible).Combuster wrote: As for directories, speed is terrible finding one directory's contents - you'll always have to run over the entire index. That and the redundancy in filenames makes it a horror to work with directories
About directory redundancy: I agree with you in this matter. If I haven't misunderstood the discussions in the original thread, the point of having the full path name in the file name was to avoid directory entries completely. However, someone pointed this scheme would not be able to represent empty directories, and their entries were added, but the full paths stayed. I can think of several ways to address this:
- Remove directory entries and keep full paths. Empty directories are represented by file entries either with their name and a trailing slash (so the entry for folder a/b/c is "a/b/c/") or by a phony entry with a special name in that directory. This brings a new problem: when creating a new file into an empty directory, its entry must be removed, and vice versa when removing the last file of a folder.
- Keep folder entries and remove full paths. The index of the parent directory entry of a file or folder is stored in a 4 byte field (again, I think that 4 billion entries is enough: FAT16 allows 65k files, while FAT32's max is 268 million). Files or directories in the root point to entry #0. I can't see any problem with this solution, so it is the one I would prefer to be included.
That was the whole point, I think ;DCombuster wrote: Apart from that, SFS is the perfect fs for those wanting to have something easy without worrying about complex algorithms, and not have windoze mess with it. definately a good experiment for newbs on the subject.