とあるOS [ToAruOS] ~ A Learning Project

[klange]

I've been here for about a year and have posted a number of screenshots of this project as it has been developed, but never had the confidence necessary to post a proper announcement thread until now...



とあるOS (pronounced 'toe ah roo OS' and meaning a "A Certain Operating System") is a kernel and set of accompanying userspace programs that a couple of colleagues and myself (98% me) have been writing over the past year or so (since January 2011). The project is a learning experience, primarily focused on understanding POSIX and general operating system design. If you're looking for an experimental or non-POSIX-ish system, this isn't one, but hopefully it's still interesting enough for you to keep reading. The kernel, which is relatively lightweight and currently monolithic, supports POSIX file I/O (with a read-write-capable EXT2 driver), ring-buffer-backed pipes (that are reasonably speedy), a complete shared memory model (though lacking in a permission system), old-style POSIX signals, processes and threads as well as a few other minor things.

I started working on the project while I was on a train from Cleveland to Chicago. I happened to have had a copy of the Intel systems manuals on my Kindle and my laptop in front of me, and had a long day ahead of me, so I started writing a kernel... I'll skip the history for the first year, as it's rather boring (I made a kernel shell and ANSI-capable terminal, which was kinda fun). In August 2011, the project become an official part of the University of Illinois Urbana-Champaign ACM student chapter's Special Interest Group on Operating Systems (UIUC ACM SIGOps for short).

The past three months have been spent building a windowing environment on top of our shared memory model in preparation for a university open day ("Engineering Open House"). I think we were rather successful:



The windowing system uses shared memory buffers for window textures, which are then rendered with a speedy hybrid stacking/compositing renderer (moving/creating/removing a window will force a rerun of the stacking algorithm, but renders to the screen happen in a draw loop ensuring that any changes made to window buffers is always accurately reflected by the display) in software. Eventually, if I have the time, I will write some real display drivers for the Intel 965 series with actual blitting, but for now we support VESA (poorly, via broken software emulation of BIOS calls; I have never had this work on real hardware because of how limited our third-party emulation layer is; I am working on replacing it with libx86emu) and the Bochs/QEMU virtual card.

My current goals for the next year include:

• Finishing the native GCC port - There are a few POSIX functions we don't yet support, mostly related to ioctl and file permissions.
• Porting bash/etc. - Similar to GCC, there are just a few things we don't yet support that we need for these.
• Networking support - Self-explanatory, I would hope...
• Building a GUI toolkit - Our windowing environment only supports passing keystrokes to windows right now and has a limited library for rendering window borders/titlebars.

The code is all licensed under the NCSA license (it's a lot like the BSD license if you're not familiar with it) and is available on GitHub. If you want to report stupidities or general bugs that would be nice. If you want to help develop anything, there are much better projects to work on here but we'll still begrudgingly accept you. If you want to run it on real hardware, good luck as our VESA BIOS emulation doesn't actually work on any non-Sea BIOS I've ever encountered and we don't support VGA text since roughly 0.0.4 (I have only ever gotten things to boot graphically on QEMU, but you should also have similar luck with bochs). Build requirements are a newer Clang (I'm on 2.9) or a reasonably recent GCC, genext2fs, yasm, and (if you want to generate our abysmally outdated documentation) a complete LaTeX distribution with CJK support. Unfortunately, we don't have a proper userspace toolchain available but once we have a working GCC port this will probably change, in the meantime, there is a script in the `util/` directory that will pull the newest set of pre-compiled binaries from Dropbox (run `util/grab-binaries.sh`).

I think that basically covers everything I wanted to put here for now. I'm going to post my updates in this thread as I work on new features and port more software instead of continually spamming the "Screenshots" thread.

[gravaera]

Yo:

Very nice, and congratulations

I hope your project continues to fulfill you hereafter as well. Happy hacking :O

--Good luck
gravaera

[eino]

Looks amazing! Can't wait to try it out asap.

[klange]

I made a clock.

[ACcurrent]

OMG! I might actually be able to use this OS in the near future!
Great Job!

[klange]

OMG! I might actually be able to use this OS in the near future!
Great Job!

Ha. Let's see if I can get it running on real hardware before we jump to declarations like that.

Unfortunately, the next few weeks probably won't see many updates from me as I'm nearing the end of my college career and approaching my last set of final exams. Perhaps I'll get some important features implemented while I'm in 広島?

[klange]

I actually bothered to set up a VGA text-mode terminal again (in userspace as a pure extension to my existing terminal, so full support for parsing all the great ANSI stuff I already supported, just not so much displaying it), built some stripped binaries, turned that all into a fresh ramdisk image, and booted on real hardware for the first time in many months. I'll try to get my real hard disk working at some point (I don't do anything with partition tables at the moment, so that's probably the first thing on my list) and completely drop ramdisks, then start working on porting a better x86 emulation library to do VESA calls with and hopefully get video output on real hardware.

In the meantime, more nyancat, this time in glorious VGA text-mode colors:



And my shell running on my laptop: (I only crammed a few binaries into the ramdisk)


In GUI-land, I'm working on adding mouse events so I can make buttons and such. I'm collecting mouse up / down / move events sanely, just need to start sending them out to the clients.

[klange]

Mouse events!

[klange]

I'm starting to work on a GUI toolkit similar to GTK, focused on function pointer callbacks and semi-efficient rendering. I have buttons with labels in a prototype application.



I have plans for a full set of widgets, from labels and buttons to scrollregions, sliders, checkboxes, etc.

[ACcurrent]

Please tell me you arent using GLib or some object system similar to GObject (yuck). I hope it supports C++ features like classes and so on. I would have prefered if you said something like cocoa or even Qt. Anyway best of luck! your OS project is amazing!

[klange]

Please tell me you arent using GLib or some object system similar to GObject (yuck).

Nothing like that, no.

I hope it supports C++ features like classes and so on. I would have prefered if you said something like cocoa or even Qt. Anyway best of luck! your OS project is amazing!

I'm still debating whether I want to do a pure C99 toolkit with manual vtables, or if I want to just bite the bullet and go for C++ (I've been avoiding C++ because my binary loader doesn't execute startup code and my PI/O disk drivers are slow enough that loading a C++ binary is not a fun experience when debugging is enabled). The prototype I'm building is pure C.

I want to port Qt at some point, but part of the "full experience" of what I'm doing is writing my own toolkit. It's fun! Trouble with Qt is, since it's a complete framework, there are a lot of complications in porting it to a new platform (just look at the Wayland port). Sort of need to get my toolchain ported first, but I've been lazy about getting simple things implemented. It's final exam season, this is my last semester of university, and I'm planning trips abroad and moves across the country, so it's a rough time for OS development. Kernel hasn't seen major changes outside of additional boot arguments in a few weeks...

[klange]

I just finished working on some scripts and patches that should be able to build a toolchain that targets とあるOS, and I'm looking for people to help me test it out.

The bit of bash below should (on a Debian derivative; just make sure you have the listed packages installed if you're on something else), from my git repository, ensure that all dependencies are installed and then build the toolchain, userspace applications, a hard disk image, and finally the kernel (which it should then attempt to run with qemu).

Code: Select all

sudo apt-get install clang yasm genext2fs build-essential wget libmpfr-dev libmpc-dev libgmp-dev qemu
pushd toolchain
./prepare.sh        # Grabs specific versions of GCC (and the libraries it depends on, though those don't get built yet), newlib and freetype, and applies patches
./install.sh        # Configures and builds GCC, Newlib, and Freetype
. activate.sh       # Adds the toolchain binaries to your $PATH
popd
pushd userspace
make                # Builds the userspace applications
popd
make system         # Builds the kernel.
make run            # And runs it with qemu

I'm mostly interested in making sure I have all the dependencies noted. I tested the scripts already from a fresh clone of the repository, but I'd like more thorough evaluation.

[klange]

Grab a VM image of a debian system and work through it from a clean, no deps installed state?

I'm doing that, but I know too much about my build system for my own good; I want to make sure someone else can work their way through it.

[miker00lz]

this looks really good! i'm going to give it a try.

[eino]

Anything new on this one?