Measuring the progress of my OS

[coderTrevor]

Recently I took an operating systems class which rekindled my interest in my hobby OS. I managed to port Doom, implement multitasking, and started writing a GUI and getting programs to display their output in windows.

I've been feeling pretty good about that.

With all this progress, I was wondering how big my OS has gotten, as I do on occasion. I would usually look up some PowerShell command to count lines but I didn't want to keep doing that. Plus we all know lines of code isn't a good metric. But it IS a metric. I mean, I think it's safe to say an OS with a million lines of code is doing a lot more than an OS with a thousand lines of code.

That got me thinking, what IS a good metric for OS development? Lines of code can fluctuate; sometimes when I'm improving something my number of lines actually decreases. So is there any metric that's reliable?

Then it hit me: the more my OS progresses, the more I add "TODO" and "HACK" to the comments of my code. So I decided to write a little AutoHotKey script that counts these very reliable metrics. The script is far from perfect but I thought I'd share what it finds anyway.

Here's how my OS measures up:

So my fledgling OS has 205 TODO's and 21 HACK's. Not bad!


Just for fun, I thought I'd compare that to some other open-source operating systems.


First up is an OS I can't pronounce but have been watching for a while:

Only 13 TODO's? I was really surprised; I thought toaruos was further along than that! I guess Klange has some work to do.


Next is another OS I've seen on the forums here:

Fun fact: managarm is an anagram for managram. With 583 TODO's, this OS clearly has potential.


If you're not aware of Serenity you really should be. The momentum this project has is incredible!

but with 561 TODO's it's only about three times as good as my own OS. It's good to know that I can catch up!


Now for the poster-child of open-source operating systems:

1,355 TODO's is not too shabby at all!


Finally, my personal favorite, ReactOS:

A whopping 4,996 TODO's demonstrate that ReactOS is easily the most advanced open-source OS, nearly four times as advanced as Linux!


Well I hope you had fun as much fun as I did analyzing some projects.

Management-types take note: TODO's is the new hot metric you should all be using!

[nexos]

That system is great! The only issue is that some OS developers (myself included ) don't put TODO comments in my code. I think the same goes for Toaruos. But that is a great system! By the way, where can I get that line counter app of yours?

[Rhodez]

I also love what you did there. Nice script.
Quite funny to measure in that way. I still think it is hard to say what is a good metric, but certainly this gives an idea of the progress.

[bzt]

I think it's safe to say an OS with a million lines of code is doing a lot more than an OS with a thousand lines of code.

That's a bad assumption. I've spent a year to drop SLoC around 6000 in my kernel for example. I often refactor and rewrite parts to use less lines (because K.I.S.S.). This doesn't mean my OS can do less, on the contrary. I often find that rewritten code can be reused more places (thus further shrinking the code and the SLoC) and even places that I haven't implemented before.

Take a look for example my bootable disk image writer, USBImager. It has very minimal SLoC, the compiled file is about 256 KILObytes, yet you can't say it does less than balenaEtcher which is 400 MEGAbytes in size (haven't checked the actual number of lines, but I'm pretty sure 400M has magnitudes more source lines than my 256k code). Both support raw images, zip, gzip, xz etc. compressed images. And my lot smaller code can actually do more: it can also send images through USB serial ports, something that the other imager can't do, and mine has a lot more translations (17 languages at the moment) in that 256k...

Or another example, my udisasm. It can disassemble ARM8v2 in 65k (about 2100 SLoC), while Capstone even in "diet" mode is more than half a megabyte (and with bigger code size goes lot more SLoC too. According to github, only the two files cs.c and AArch64/AArch64Disassembler.c are more than 3000 SLoC, and there are about 20 source files more for that arch only). Yet, you can't say that Capstone is doing more, actually it does less in "diet" mode than my disassembler.

You probably heard this before: "Simplicity is the ultimate sophistication".

Then it hit me: the more my OS progresses, the more I add "TODO" and "HACK" to the comments of my code.

This makes no sense. The more an OS is closer to its final stage (more progressed that is) the less TODO it should have. Ideally a fully progressed and ready to use OS should have exactly zero TODO lines left.

Just for the records, these are not useless, both TODO list and SLoC are very useful, just not good for measuring the progress. For example my build system automatically generates TODO.txt (currently I have 8 ), and I also have an SLoC calculator.

Cheers,
bzt

[klange]

- Line counts are a dubious measurement of anything, and language and code style must be considered when comparing them even for a base level of "how much work went into this". The best engineer I ever knew had a net negative line count during her tenure at the company we worked for.
- I use XXX a lot for "TODO/HACK" comments, you'll find 77 of those if I've done my counting right, a bit more than the 13 TODOs. Also I use an issue tracker and a physical notebook to track projects. TODO comments aren't great, and the idea of using the number of them as a positive indicator of progress seems a bit paradoxical to me. If you have a lot TODO you must not have done much.

Progress should better be measured in features and capabilities.

[mrosdev]

Amazing!

[bloodline]

Fascinating metrics, but unfortunately they don’t tell us very much, and certainly can’t be used to compare projects.
One would need to use proper project management tools and techniques for that.

Apologies for butting in, I am an Analyst in my day job.

Great work on your project though, that’s worth a drink