Server OS in ten seconds flat

[turdus]

@brain,@solar: yep, public release wasn't intended originally. Parse_header is just gets the job done, you can rewrite it if you like. I had 4 hours to do it, that's why it's so ugly. The point is, why I've pasted the code is in mini_httpd() and http_request(). They are commented, and you can get a clue what's wrong from the log messages pretty well. I wanted to show you it's not so hard to create a simple webserver as everybody thinks; dependency hell is not the only possible way to go.
@berkus: yes, it is, behind two layer7 firewalls and at the end of a vpn tunnel, serving queries from only one specific form, used by only trained and authorized persons Beside of that, go on, tell me how would you hack it.

[turdus]

Well, feeding it a never-ending HTTP header sounds like an interesting option.

Good point, but that will never get pass through the layer 7 firewall. The specification was clear that the code has to deal with RFC compliant and not-oversized http headers only, that's why I wrote "it's not stupid-proof (not much error handling)".
But to answer your question, here's a quick patch:

Code: Select all

#define HEADER_MAXLINES 1024
            while(1){
               http_req=realloc(http_req,strlen(http_req)+1025); // Never fails
               memset(http_req+strlen(http_req),0,1025); // Neither
               if(++header_lines<HEADER_MAXLINES&&fgets(&http_req[strlen(http_req)],1024,stdin)!=NULL){

You can put it in the line before, if you don't prefer the compactness of C
And another note, before you ask: sys_log will exit the thread after logging error or critical message. If you don't like compact code, or code size does not count to you, you can replace

Code: Select all

sys_log('E',...)
->
{ sys_log('E',...); exit(1); }

everywhere.

[NickJohnson]

Hi again,

We've settled on a few more details of the design:

First, the system is not going to be 64-bit. Instead, we're going to recycle some of the Rhombus microkernel as a base for the new monolithic kernel (just the paging, multitasking, and threading components.) As a result, we are sticking to just 32 bit, and we have a slightly more interesting kernel design. Instead of a typical monolithic blob, the kernel is going to be divided into two (monolithic) layers, one running in ring 0, and one running in ring 1. The ring 0 component will basically just be the Rhombus microkernel minus IPC and process management, while the ring 1 component will be all drivers and userspace-accessible system calls. This separation is not (at least primarily) for protection reasons, because ring 1 can still access kernelspace memory. Instead, this will allow us to do preemptive multithreading of the ring 1 layer very similarly to the ring 3 layer, because the TSS handles interrupts from ring 3 and ring 1 similarly.

Parts of this are already complete. When the interface to the ring 0 layer is complete, we may release it separately as a base that others could use (since it defines pretty much nothing about the behavior of the rest of the kernel, yet does a lot of the heavy lifting.) The full sources still won't be released until the end, however.

[ACcurrent]

Mongoose is a good HTTP server that has few dependencies and can handle POST, GET, etc.

[pcmattman]

Sounds like fun, I look forward to seeing what you all come up with. Keep us posted

[NickJohnson]

So, it's been six months.

The project was going well for the first 3 months, and I got a lot of things working, but I can't claim that I properly finished. I got bogged down with school and work and other things after summer ended, and fell way off track. The friend who was helping me also ultimately did very little, and I was sort of counting on having some help to get all the pieces working.

At this point, it has enough of a TCP/IP stack working that it can serve basic HTTP requests (purely from kernelspace), can switch to and from usermode, and load files from a CD. Not too shabby for what was effectively 4-5 months of work, but a far cry from a proper UNIX-like system.

Fortunately, I can salvage some good pieces because of the design. The Pinion project is a mirror of the piece of the system that runs in kernelmode and provides threading, paging, and interrupt handling facilities to the rest of the kernel. It's not super-advanced, but it's stable and likely useful as a base for other systems. I'm going to be updating the github repository with the latest version today or tomorrow.

So, here's what I'm going to do: try it again. I've learned a ton about TCP/IP, and have ironed out most of the kinks in the funky Actor-model modular kernel design I was trying to build, and I think I can get it to work this time. I'm going to take an official break for a couple of weeks to get some things in order, then start with a fresh fork of the Pinion base. However, the ultimate goal has changed a bit: what I'm really trying to build is a system that:

- has a full TCP/IP stack
- can serve dynamic webpages
- is modular and extensible, and makes it easy to write concurrent code
- is highly robust/fault tolerant even running C code in a single address space

Whether this is a proper OS, a webserver running on an exokernel, a low-level asynchronous server framework, a standalone networking toolbelt, or just a cool toy may become blurry, but I want to build it essentially from start to finish in at most six months. Otherwise, it's just no fun.

[eino]

Well at least I'm waiting for the new Pinion. Hopefully upgrading won't be too painfull.

[NickJohnson]

The Pinion repository on my github has been updated. The sample code for the system part has changed considerably, but the thread model and general architecture are pretty much the same. If you need some advice about upgrading, just PM me.

[box]

Exciting! I can not wait to monitor this project's progress.