Relatively, how complex is writing a ext2 driver?

[r0nk]

Relative to other parts of your standard unix clone I mean. I've recently got my ext2 driver "working", and I read on the osdev wiki that its a somewhat complex filesystem. I'm wondering how hard the other parts will be compared to it, as the ext2 driver was the first thing I did after interrupts. Writing a simple memory manager was a breeze (although its not a great mm), so i'm wondering how high the other mountains are now that I've climbed one.

[Kevin]

I'm not primarily writing a "standard unix clone", but the ext2 driver to which I contributed about half (the CDI one) has 2.7 KLOC. The rather minimalistic memory management code of the "standard unix clone" is at 1.2 KLOC, the more serious, but still simple MM of tyndur takes 2.1 KLOC. Of course, that doesn't say much about the difficulty, but if you see that one component is much smaller (or much larger) for you, that might already tell you something.

My experience with ext2 was that it's quite easy to get something up and running that can read some files from the filesystem. The hard part was writing to the filesystem without breaking things. And that's not necessarily obvious breakage for every single write, but it can be just some corner cases that introduce some data corruption. If you don't have lots of test cases, it's easy to mess up without even noticing at first.

So I guess the answer to your question depends on another question: Have you implemented just something, or something good? If it's the former, you're not done with it, you just don't know yet. If you have something good for both memory management and a filesystem, the other mountains won't be much (if at all) higher, but they will still be numerous.

[Candy]

It is also a major difference if you're doing a read-only driver that can assume the FS is in good state, or whether you're making a read-write driver that can also modify (and mess it up) and then cause the reading to fail subsequently. I've written a few read-only drivers and those are (imo) a piece of cake to do.

[sortie]

I expect that if you can do an ext2 driver and consider memory management to not be that bad, you shouldn't be too worried. It's still a lot of work doing new, larger tasks, but your ability to deal with such tasks matters more than their actual size. Writing an ext2 driver requires you to comprehend a lot of concepts, semantics, data layouts, have good deal of experience, and so on. An inexperienced newcomer without that much experience would find plenty of ways ext2 would be a task they can't do.

In my experience, I tend to overestimate how much work a useful larger feature actually is to implement, and I put it off implementing it for way too long and instead do something worse but simpler, which I regret doing when I finally do the real thing.

[r0nk]

I tend to overestimate how much work a useful larger feature actually is to implement, and I put it off implementing it for way too long and instead do something worse but simpler, which I regret doing when I finally do the real thing.

Yeah, but one of the things I'm learning through osdev is that It's really difficult to do things right the first time, and that it's usually easier to write things the easy but wrong way and then improve/fix it, then it is to get it perfect on the first try. Perfect on the first try is often an overwhelming task.

So far it is a read-only file system, I figured that writing wouldn't be hard. Guess not D:

In spite of the difficulty, I'm learning so much so quickly about the way things work, its incredible. I should've tried OSdeving a long long time ago.

[Alexis211]

I'm not quite sure about writing easy/wrong implementations first and improving later, as you will probably have to change all your architecture later on when you understand how to do things properly. There is kind of a distinction between how your code is structured (function names & prototypes, class hierarchy if you do OOP, VFS interface if you do a VFS, and so on), and the actual implementation. You can have a clean architecture and a lot of functions that just fail, printing out a "not implemented" error. I'd recommend you think about how you would go if you were to do the full implementation, and postpone only the parts that are too complex and not needed immediately.

About the ext2 fs, I can't tell you how complex it is to implement (I've never done it), but along with FAT it's probably one of the simplest filesystems out there.

[sortie]

FAT has so much basic brain damage in its design that I very much recommend entirely skipping it and going directly for ext2. For some reason, I personally find it much easier to comprehend real designs than broken designs.

[r0nk]

FAT has so much basic brain damage in its design that I very much recommend entirely skipping it and going directly for ext2. For some reason, I personally find it much easier to comprehend real designs than broken designs.

I was going to go for FAT first but when I read that its header has a instruction in it I was like yeah no.

[Brendan]

Hi,

FAT has so much basic brain damage in its design that I very much recommend entirely skipping it and going directly for ext2. For some reason, I personally find it much easier to comprehend real designs than broken designs.

FAT is used for all sorts of things (including devices like digital cameras, etc; and including UEFI's partition) - it's mostly a "de facto" standard for exchanging files between different systems (and the fact that it's mostly bad doesn't change the fact that it's something an OS is likely to continue using). In comparison, ext2 is obsolete and isn't used for anything.

However, it's not quite that simple.

Mostly for file systems there's 2 categories - those that are good candidates for exchanging files between different systems (FAT, ISO9660); and those designed specifically for a specific OS's main file system (NTFS, HPFS, ext3, ReiserFS, zfs, etc). For the latter, the file system is designed to accommodate the OS's own native file permission system and the OS's own special features (e.g. resource forks, snapshots, etc), which makes them completely unsuitable for a different OS.


Cheers,

Brendan

[Brendan]

Hi,

Relative to other parts of your standard unix clone I mean. I've recently got my ext2 driver "working", and I read on the osdev wiki that its a somewhat complex filesystem. I'm wondering how hard the other parts will be compared to it, as the ext2 driver was the first thing I did after interrupts. Writing a simple memory manager was a breeze (although its not a great mm), so i'm wondering how high the other mountains are now that I've climbed one.

If I have 2 pieces of string, which one is the longest?

Ext2 can be very simple (e.g. synchronous, read only, no caching) and it can be very difficult (asynchronous, high performance including taking into account the underlying device's performance characteristics, special ordering of writes and synchronisation/flushing to minimise corruption if power fails, features like optimising data layout and checking in the background, more features like honouring per-user disk quotas and maybe per-user encryption and maybe things like "secure erase", etc).

In the same way other parts of an OS can be very simple (e.g. basic round-robin scheduler) or very difficult (e.g. "O(1)" real-time scheduler that's been heavily optimised for large many-core NUMA servers).


Cheers,

Brendan

[Candy]

> If I have 2 pieces of string, which one is the longest?

When comparing Stephen Hawking's life and the concept of serendipity, which is the most purple?

[alexfru]

I was going to go for FAT first but when I read that its header has a instruction in it I was like yeah no.

That instruction (the 2-byte short jump at the beginning of FAT boot sectors) doesn't have to be executed. It's just that for whatever reason some of Microsoft's code checks for it and if it's not there decides that this is not a valid FAT FS.

The instruction does not tie one to x86 systems or DOS/Windows or anything. Just have it in there and forget about it. You can't do much with 2 bytes anyway.

[thomasloven]

I wrote a read-only ext2 driver in about two weeks back in 2010. Since then, I have been working on adding writing...