Baby Steps: Starting Modestly?

[NetBSD-User]

Did you start your O/S with a wishlist of features looking like a modern UNIX system, and set about implementing them from day one, or did you start more modestly (e.g. DOS-ish?)

Me: unsure, started but killed project when I needed to learn a hell of a lot more about kernel development before I took on anything more ambitious than a DOS-like O/S. Making extensive modifications to Minix 2.0 to learn more first.

[Pype.Clicker]

well, imho, deciding to start "DOS-ish" will prevent you from reaching a fully-featured mature level.

[Solar]

Well, I had a wish-list, mostly inspired by AmigaOS, adding some niceties missing there, and noting down all the things that I wouldn't want to do as in Unix or Windows - and overall, very ambitious.

I took that wish-list, and invited other people to work on it, flesh it out, expand it. That went for 1 1/2 years, and made that wish-list even more ambitious.

I felt it was the right way to do things, because your building can only be as stable as its foundations. (See DOS / Win3.1 / Win95 / Win98 / WinME.)

Then, I started implementing, and suddenly all those so very much interested in demanding things be done their way vanished, probably because it started to smell like work.

In the meantime, the project has been scrapped altogether, and I focus on working on the OS FAQ and a C library for the people here.

[distantvoices]

wishlist ... nay, I haven't setup anything as ambitious as "learning OS development".

In the beginning it 's been nuthin' more than: "I want to start something from floppy - letz see what I can achieve" and much reading and trying out.

've never had a hankering to lure others into my doings. I do 'em alone, and thats it.

stay safe

[crc]

I have a wishlist of sorts. But it's a far cry from UNIX

Basically I wanted a single-tasking, totally open 32-bit OS with support for the hardware I use on a regular basis. Open meaning that I have full access to the underlying hardware. What I currently have (after several failed attempts) is a Forth-based OS kernel and a few applications. It already meets my basic needs and has support for a few of my desires (like persistant storage of data). Eventually I'll finish up the driver framework (with things like autodetection/compile on demand) and code a few more drivers for it. Once that's done it'll be exactly what *I* want to use on a regular basis.

[NetBSD-User]

I re-started working on Minix because it's stable, feature-full for such a little O/S, and has obvious room for bloat -- er -- improvement.

I went nowhere starting from scratch. I worked for years. It is just so hard to read obfuscated standards specs and turn it into code. It's so hard to write drivers for undocumented or poorly documented hardware. It is hard not being able to debug over a null modem cable to a second machine. A rush of kprintf messages is no good when there's hundreds of them. And when you don't have a filesystem, logging to /var/log/kernel is no good.

*Sigh* If I weren't so stubborn and determined to have me some fun writing an O/S I would have given up enough times to have wasted a quarter of my life.

[NetBSD-User]

I have a wishlist of sorts. But it's a far cry from UNIX

Basically I wanted a single-tasking, totally open 32-bit OS with support for the hardware I use on a regular basis. Open meaning that I have full access to the underlying hardware. What I currently have (after several failed attempts) is a Forth-based OS kernel and a few applications. It already meets my basic needs and has support for a few of my desires (like persistant storage of data). Eventually I'll finish up the driver framework (with things like autodetection/compile on demand) and code a few more drivers for it. Once that's done it'll be exactly what *I* want to use on a regular basis.

That's very interesting. I actually want a Lisp machine. I think there's great personal potential here because I feel so much more expressive when coding in Lisp. (And I usually am. I feel in assembler world when coding in C.)

What I had planned was a little kernel, similar to a microkernel, that does the barest basics of initialising and then executing Lisp code with its in-kernel Lisp machine. Process scheduling, memory management, device drivers, everything would be done in Lisp. Of course, some would dig down into assembler by necessity, but not much more than stuff like:

(defun inb (port)
(__asm IN #port) )

or whatever.

I am not ready for such an ambitious project just yet, but it's twinkling in my mind's eye...

[Pype.Clicker]

well, to make my point more relevant, when i started, i had not much thought about how programs would set up their address space so that i can load libraries, etc. later. As a result i had a very efficient (in both memory and time) allocator for virtual addresses, but it was almost completely useless. I took about 2 month to re-design something valuable, 6 month to re-implement it and restore code that was based on the old interface...

[Brendan]

Hi,

Did you start your O/S with a wishlist of features looking like a modern UNIX system, and set about implementing them from day one, or did you start more modestly (e.g. DOS-ish?)

I first started with OS's after buying a 4086-33SX running DOS/win3.11. I began my obsession with a real mode single tasking OS with everything (apps, kernel, drivers, etc) in one binary. Since then my wish-list has evolved (now it's huge).

Me: unsure, started but killed project when I needed to learn a hell of a lot more about kernel development before I took on anything more ambitious than a DOS-like O/S. Making extensive modifications to Minix 2.0 to learn more first.

There seems to be a general learning process involved with OSs:

Step 1: Start writing (or re-writing) the OS to the best of your abilities
Step 2: Realize the limitations of what you've done
Step 3: Return to step 1 with more knowledge/experience

Making extensive modifications to Minix 2.0 seems like a good alternative (or addition) to this cycle...

In the past I've "re-written" my whole OS every couple years, with each rewrite being more ambitious and ending with much better code/features than the last version.

For e.g. my current rewrite was started because I wanted to add support for PAE, page colouring and multi-cpu (and later decided to add NUMA) which seemed too much to add to the existing OS (which wasn't designed for these things).

I think (hope) I've reached the point where I've done everything enough times for the current rewrite to be the final rewrite

well, to make my point more relevant, when i started, i had not much thought about how programs would set up their address space so that i can load libraries, etc. later. As a result i had a very efficient (in both memory and time) allocator for virtual addresses, but it was almost completely useless. I took about 2 month to re-design something valuable, 6 month to re-implement it and restore code that was based on the old interface...

I've spent the last 6 months rewriting my boot code! Seems like ages, but if you saw it you'd realize why... I'll release it soon and then start rewriting the kernel to match it


Cheers,

Brendan

[Candy]

For e.g. my current rewrite was started because I wanted to add support for PAE, page colouring and multi-cpu (and later decided to add NUMA) which seemed too much to add to the existing OS (which wasn't designed for these things).

Hmz That's good news. My current rewrite (my first rewrite actually, second write) added PAE and page-colouring, and a few hooks (thoughts) for multi-cpu. I wanted the multi-cpu in a module (which is possible) so I decided to postpone it until I have started up the module system, which took me the past 8 months and my breath 2 days ago (the harddisk with the only backup started scratching the surface audibly). All files but 2 were saved however (total of 1200 files, so that's an awful good score) and I'm still rewriting my own boot loader too.

I've spent the last 6 months rewriting my boot code! Seems like ages, but if you saw it you'd realize why... I'll release it soon and then start rewriting the kernel to match it

Spending the past 8 months writing module loading code that loads modules from a memory-bound module archive of my own making (tar is crap and the rest didn't work). Also only going to test it once I get this 64-bit code worked out and compiling. Just like you

[distantvoices]

Well - I'm rewriting *parts* of code. Usually, when it comes over me, I take a whole module and smash it in pieces to come out with something better - just because I consider the current state of the module a load of crap.

To avoid misunderstanding: under module I understand a single *.c file (which usually results in an object file after compiling)

The next module I'm gonna rewrite is the TCB management and scheduling.(called "process.c") Current state is developped with inspiration by Minix - which has taught me a lot - but it lacks the elegance I want to have. So - take the hammer and smash it.

Recently, I've rebuildt the whole gui service - as it 's been one single *.c file and this one reached the 4000 lines margin - I've decided to split it up in smaller modules and units - the lions share (window management and window clipping subsystem) is still in one file, but it is only 1600 lines. Well - the modules are easier to debug and it is easier to build some object oriented framework for controls and containers.

@candy: module loader? do you intend to add object modules with a predefined interface at runtime so one can say "get_interface" to obtain a pointer to the table of functions for a loaded module? Does this imply runtime linking and function table patching? Me s curious coz I wanna have something similar for my services(fs,gui,...) in order to make them runtime extensible.

[Candy]

To avoid misunderstanding: under module I understand a single *.c file (which usually results in an object file after compiling)

The next module I'm gonna rewrite is the TCB management and scheduling.(called "process.c") Current state is developped with inspiration by Minix - which has taught me a lot - but it lacks the elegance I want to have. So - take the hammer and smash it.

Recently, I've rebuildt the whole gui service - as it 's been one single *.c file and this one reached the 4000 lines margin - I've decided to split it up in smaller modules and units - the lions share (window management and window clipping subsystem) is still in one file, but it is only 1600 lines. Well - the modules are easier to debug and it is easier to build some object oriented framework for controls and containers.

We must have a different definition of small and big. If one of my C files is bigger than say 200-300 lines I split it up, unless it is repetitive code. All of my modules are subdirs to allow this easily, and have a hierarchical makefile that is simplified to only provide parameters to a default makefile (makefile.default) that fills in the gaps and is such a piece of crap that I hate modifying it . Still, all the other makefiles are smaller than 8 lines, and consist of 2 includes, 2 blank lines and up to 4 variable definitions (not more complex than OBJECTS=bla.o in.o out.o).

@candy: module loader? do you intend to add object modules with a predefined interface at runtime so one can say "get_interface" to obtain a pointer to the table of functions for a loaded module? Does this imply runtime linking and function table patching? Me s curious coz I wanna have something similar for my services(fs,gui,...) in order to make them runtime extensible.

Well... predefined is a big word. The only thing predefined is that their file must contain an aosmod section (yes, ELF is required), and that section defines the indirect calls. Other than that, it's pretty much runtime linking and loading, probing the module with all objects that are currently available of a type that it might understand (so I don't offer all objects to all modules, saves probe time and thus speed), and it then releases new objects. A few types are predefined, these interface with the user API call interface. That's pretty much all that's defined, the module might create its own address space, or just be confined to it, for safety's sake.

This also explains why I can't call my kernel monolithic (it's in a load of files) and I can't call it microkernel (since it allows, and usually encourages, loading drivers in kernel space). It has such a mix (imo) that it can do the good parts of both.

[mystran]

That's very interesting. I actually want a Lisp machine. I think there's great personal potential here because I feel so much more expressive when coding in Lisp. (And I usually am. I feel in assembler world when coding in C.)

You are not the only one. I'd love a Lisp machine too. Specifically one that used a dialect of Scheme (ie, tail recursion and continuations, but with Common Lisp -style macros). And as for C, it IS the world's most portable assembler.

As for programming with the "lesser languages" after being enlightened by Lisp, Python is almost as nice (you just miss the macros) and if you know C++ well (especially templates and such) you can make it do quite some nice things (and operator overloading can even help with the syntax a bit..)

[Solar]

This also explains why I can't call my kernel monolithic (it's in a load of files)...

"Monolithic" doesn't mean "one single binary image", but "all features running in kernel space", AFAIK.

[Dreamsmith]

This also explains why I can't call my kernel monolithic (it's in a load of files)...

"Monolithic" doesn't mean "one single binary image", but "all features running in kernel space", AFAIK.

Well, having it in a bunch of files doesn't make it not monolithic, but locating features outside of kernel space doesn't make it necessarily not monolithic either, nor does having all features running in the same address space necessarily make it monolithic. A single address space system can have a microkernel, and pretty much all modern operating sytems with monolithic kernels have some OS features located outside of kernel space.