SimpleFS - Missing from WIKI and Brendan's site

[JamesM]

Linux uses devfs/udevd/procfs and the like, which basically mounts a different filesystem under a different mountpoint. Following that analogy you could just reuse the VFS structure and put devices where they can be put and use SFS for the root.

Besides I don't think its a good idea to taint SFS with unix-specific properties since that goes straight against its principles.

I know how linux does it. In the initial stages of boot the initrd is loaded, which has handmade character and block devices in its /dev directory. It uses these until udevd/devfs is loaded and pivot_root called.

I wanted to avoid having a seperate devfs filesystem in the tutorial series to simplify things - I suppose I have no choice now but to implement a devfs. Oh well...

James

[Brendan]

Hi,

I'm rewriting my tutorial series from scratch, and wanted to implement a "real" filesystem as one of the chapters. SFS really seems exactly what I'm after - I'll write a set of command-line "mtools" style tools to deal with it too - however as the tutorial series creates a UNIX-like OS there are a few things SFS is missing, namely character and block devices.

Sockets and named pipes I don't really care about, but char and block devices are fairly essential. There seem to be a number of unused index area entry identifiers - would an addendum that added these two filetypes to the specification be acceptable? I'm happy to draft it up.

For Unix, there's a Virtual File System and a variety of different file systems that are mounted at different places in the VFS. Some things (e.g. sockets, named pipes, anything in the "/dev" directory and anything in the "/proc" directory) exist in the VFS, but don't exist in any normal file system (they don't exist on any storage device).

For the "/dev" directory and the "/proc" directory there's actually special purpose "file systems" in the kernel (e.g. "devfs" or "udev", and "procfs"). These aren't real file systems and none of these files exist on any storage device.

For a very rough example, imagine you wanted a "/proc/BIOS_mem_map" file for your kernel. When someone attempts to write to this file you ignore the write, and when someone attempts to read from this file you call the BIOS function and store the data returned by the BIOS into the caller's buffer.

For another example, imagine the kernel has a list of device drivers. When someone reads the "/dev" directory you make up a directory entry for each device driver and return that, and when someone opens a device you forward read/write requests to that device driver.

Mostly what I'm saying is that none of the things you want would exist in any normal file system (like ext2, reiserFS, NTFS, etc), and therefore they probably shouldn't exist in SFS either...


Cheers,

Brendan

[Combuster]

To be honest, devices have always been a special case. Unix dictates that everything should be a file, so it makes files from devices. The interface behind that talking to a device will always be different compared to talking to a filesystem will lead to the fact that you always have a case split somewhere: file or device.

Just having some mknod code which dumps some weird entries into some magic directory and have the kernel decode the meaning can be done about the same by the kernel just dictating that info. So instead of manually having to create those entries every time via some shell script, you can move the code around and let the kernel do that. You'd even have the information stored together leading to less unhappy mismatches.

Bottom line: devfs' functionality exists in some form in one place or another. So while you could hack around hiding it, you could just give the thing a name and have it more obvious how it works.

$.02

[JamesM]

Hi,

I'm slightly ill at the moment and about to go out so I'm not going to argue the point very heavily. Yes, you can special case in the kernel. Yes, Combuster's point is taken. I knew all along that I could do it that way - I just didn't want to. I will, however.

Last point: ext2 does support character and block devices, as well as sockets and named pipes. Most UNIX-oriented filesystems do, because many UNIces still require it - if you don't have a /dev/tty in your initrd, linux won't make it for you until it's mounted it's root!

James

[OrOS]

Nudge: SFS converted to wiki format and mirrored: http://wiki.codecall.net/OS:SFS

[quok]

Hi,
Last point: ext2 does support character and block devices, as well as sockets and named pipes. Most UNIX-oriented filesystems do, because many UNIces still require it - if you don't have a /dev/tty in your initrd, linux won't make it for you until it's mounted it's root!

I'll second that. The difference between it all is handled in the VFS. The VFS looks at the file type stored in the inode and if it's a character or block device, calls a different set of functions than it would have otherwise called if the inode indicated a normal file. You can find more about that in chapters 12, 13, and 14 of Understanding The Linux Kernel, Third Edition (a PDF copy of which can be found floating around the net). The whole reason ext2 and other filesystems support this is because there's nothing that says a character or block device, or a socket or whatever, must reside in /dev, and there's still some systems that don't support a dynamic /dev as well, so those entires must either be stored in the filesystem somewhere or recreated upon boot.

[Brynet-Inc]

Hi,

I'm rewriting my tutorial series from scratch, and wanted to implement a "real" filesystem as one of the chapters. SFS really seems exactly what I'm after - I'll write a set of command-line "mtools" style tools to deal with it too - however as the tutorial series creates a UNIX-like OS there are a few things SFS is missing, namely character and block devices.

Sockets and named pipes I don't really care about, but char and block devices are fairly essential. There seem to be a number of unused index area entry identifiers - would an addendum that added these two filetypes to the specification be acceptable? I'm happy to draft it up.

For Unix, there's a Virtual File System and a variety of different file systems that are mounted at different places in the VFS. Some things (e.g. sockets, named pipes, anything in the "/dev" directory and anything in the "/proc" directory) exist in the VFS, but don't exist in any normal file system (they don't exist on any storage device).

For the "/dev" directory and the "/proc" directory there's actually special purpose "file systems" in the kernel (e.g. "devfs" or "udev", and "procfs"). These aren't real file systems and none of these files exist on any storage device.

.....

Cheers,

Brendan

That example isn't entirely accurate, the concept of a special "device filesystem" is of Plan 9/System V design.. many Unix systems still use the traditional /dev/MAKEDEV script.

It's only recently that FreeBSD started using it... OpenBSD and NetBSD do not.. probably never will.

So in that case, filesystem support for device nodes is important.

(Also, BSD support for /proc is optional, and it's often not utilized..)

[JamesM]

Hi,

Given that it seems I'm not the only one "fighting my corner", I'll take the liberty of writing up a draft addendum and posting it here for review. I'm not sure how we're going to decide if it gets ratified or not - maybe a vote?

I can see windows people voting against and UNIX people voting for, and a religious argument coming out of it too, so I'm going to tread very carefully!

James

[Brendan]

Hi,

Given that it seems I'm not the only one "fighting my corner", I'll take the liberty of writing up a draft addendum and posting it here for review. I'm not sure how we're going to decide if it gets ratified or not - maybe a vote?

I can see windows people voting against and UNIX people voting for, and a religious argument coming out of it too, so I'm going to tread very carefully!

IMHO, pipes and sockets are essentially volatile (when you turn the computer off they're gone), so using non-volatile storage for these things is entirely unnecessary (excluding the swap space, I guess). I was wrong about character and block devices in ext2 - I didn't realize the Unix world was so lame (you'd think I'd know by now).

Despite this, these issues are mostly irrelevant. SFS was designed as a very simple portable file system (e.g. for transferring data between computers, without problems like patents and without compatibility problems between different OSs), and was never intended to be used as an OS's main file system. As an OS's main file system, SFS would have poor performance characteristics and lacks some significant features, like file permissions (read/write/execute, owner, group, etc), symbolic links and special file types. These design decisions were entirely deliberate - adding any of these features reduces the usefulness of SFS for it's intended purpose (e.g. for transferring data between computers without compatibility problems between different OSs).

Mostly, it's like modifying a bicycle so you can fit 3 kids inside and tow a caravan - it makes a lot more sense to use the right tool for the job from the start (or at least, avoid using the wrong tool for the job)...


Cheers,

Brendan

[quok]

IMHO, pipes and sockets are essentially volatile (when you turn the computer off they're gone), so using non-volatile storage for these things is entirely unnecessary (excluding the swap space, I guess). I was wrong about character and block devices in ext2 - I didn't realize the Unix world was so lame (you'd think I'd know by now).

I'm just going to be a bit nitpicky here and point out that it's not just ext2 that supports this, but all UNIX filesystems, including ZFS. It's part of the POSIX spec. Although I will agree that using non-volatile storage for pipes and sockets is silly. All applications that use them first test for their existance and will create them if necessary, or error out as appropriate. IMO, the pipe and socket file types could be stored entirely in RAM. However, I can see some use cases for doing it the way it's done now as well.

My whole reason for posting to this thread in the first place wasn't to say 'yea' or 'nay' to an alteration of the SFS spec so much as it was to correct a little bit of mis-information.

However, since I'm here I may as well throw in my (probably worthless) opinion. While I think the spec could be altered to add the support JamesM wants, I tend to agree with Brendan in that SFS simply isn't the right tool for the job. There's too much missing from this filesystem for it to really be useful as the primary OS in a UNIX-clone. Granted, linux does have FAT32 support and all the owner/group information is emulated and stored in memory, but it is used primarily for the same reason as SFS was designed -- for transferring data between computers.

Regardless of if the SFS spec does get altered or not, it is functionality that could simply be ignored by an implementation. An OS that doesn't support UNIX style character or block devices could simply ignore those filetypes, not even displaying them as being on the disk (as long as it didn't erase them from the disk).

James -

I'm sure you picked SFS because of it's simplicity, and simplicity in a tutorial series is good, but I'd rather see something like a ramfs, and they're rather easy to implement. If you really want to go all out, you could go for read-only ext2 support, or a fat32 driver.

[deleted]

Hey guys, the link to the spec seems a little broken any fixes?

[Combuster]

There's no reason to resurrect a dead thread for something that's long documented on the Wiki. SFS