OSDev.org

1. Microkernel with (minimal) drivers in ring1/2 and their segments appropriately assigned (I'm planning to still use sysenter/syscall stuff and I have some questions about that)
2. Minimal hardware support (specific hardware supported... don't have it, out of luck).
3. Single-process, multi-task. Multi-core/cpu necessary, but 4+ core/cpu highly recommended.
4. General IRQs firing on CPU0, drivers running on CPU0, any general task/gc on the ring0 level, and the main process on CPU0.
5. Main process "workers" are sticky to CPU1..n and there's no preemption except during panic.
6. All I/O is async by nature unless explicitly overridden.
7. 64-megs minimum RAM (including a simple ring3 process)
8. The heart of the kernel is something similar to kqueue or epoll or IO completion ports, but much more powerful and much more elegant.
9. A basic TTY and user system to allow controlling/reviewing the system, but nothing beyond that.

1. Support most environments (I already said this was a non-goal... repeating).
2. POSIX/BSD/LIBC/whatever compatibility of any kind. The antithesis of a goal.
3. Support of any standards or designs that might hinder the intention of this design (such as would cause crippling resource consumption on CPU0)
4. Conventional ring3 task switching (although semi-support for CPU0 given it's need to handle the drivers).
5. Support for things like OpenGL, any kind of window management system, stdio, what-have-you... it's all out the window. No web browsers or super-duper shell scripting or wine or whatever. You get the picture.

1. Management of all CPU0 IRQs
2. Task switching between the kernel, all drivers, and the main process thread.
3. Handling low-level errors and fataling gracefully as necessary.
4. Proxying/buffering/etc all "driver" requests (filesystem/network/etc).
5. Managing all I/O wait states in a poll() state (the heart of this)
6. Avoiding event starvation via defined distribution patterns and event preemption/aborting as necessary.
7. Provision of generic event tracking and firing.
8. Provision of low to mid-level atomic, semaphore, mutex, and consumer/producer models.
9. Panic support and recovery.

handle sockworker = undefined;
handle logger = undefined;
handle userland_event = undefined;
handle conn_consumer = undefined;

struct worker {
	// worker specific vars can go in here (really just thread-local-storage)
	handle events;
};

main() {
	// In CPU0
	handle events = events_create();
	handler logger = open(events, "file:///var/log/whatever.log", APPEND);
	// WAITFOR here does a couple of things... tells the write call to I/O block,
	// and possibly rw locks (in this case write). The workers aren't started
	// yet so contention isn't possible... so the rwlock doesn't matter.
	write(logger, "Starting up\n", WAITFOR);
	sockworker = worker_register(sockworker_setup, sockworker_teardown, sizeof(worker));
	conn_consumer = worker_consumer(sockworker, sockworker_acceptconn, ROUNDROBIN, nullptr);
	worker_startup(sockworker);
	handle listener = listen(events, "tcp://[*]:12345");
	events_on(events, PROCESS | EXCEPTION, UNHANDLED, unhandled_error, nullptr); // << just an idea... don't know where it goes yet though.
	events_on(events, TIMER | ONESHOT, 60*60*1000000, after_one_hour, nullptr);
	events_on(events, LISTENER | INCOMING, listener, do_accept, nullptr);
	userland_event = events_register(events);
	events_on(events, USERLAND, userland_event, arbitrary_userland_event, nullptr);
	status = events_wait(events, INFINITE);
	// We're shutting down for 'status' reason now... a couple of things:
	// First, if it's a graceful shutdown (SIGTERM or whatever), then
	// sockworker_teardown() has already fired and it _had_ an opportuinity to
	// gracefully shutdown (or hang indefinitely if it's a jerk).
	// An event hook like PROCESS | HANG could probably be be provided to force
	// a SIGKILL of hanging workers after x-amount of time.
};

do_accept(handle events, handle listener, void* userland /* is nullptr */) {
	// We're still in the "main" process on CPU0.
	// This _could_ be given to a worker, but it's just an accept(), so why?
	handle conn = accept(listener); //immediate so no blocking/event poll is needed
	worker_give(conn_consumer, conn); //this, however... _may_ be blocking depending on system resource limits
};

sockworker_setup(worker* wls) {
	// This is now run on CPU1..n in parallel
	// Initialize anything in wls (worker* ... worker local storage)
	wls->events = events_create();
	// Note the possible syncronization issue below... this could be handled
	// kernel-level because of the WAITFOR status (a classic rwlock).
	write(logger, "Worker online\n", WAITFOR);
};

sockworker_teardown(worker* wls) {
	// Still in CPU1..n
	// Shutdown/waitfor any active connections this worker was handling
	write(logger, "Worker going offline\n", WAITFOR);
	// Destroy anything in wls
};

sockworker_acceptconn(worker* wls, handle conn, void* userland /* is nullptr */) {
	// Still in CPU1..n
	// Possible contention again, but this time we don't care. Just try to gain
	// a lock... if not, move on without completing the action.
	write(logger, "Hey there, I've got a connection to work with!\n", FIREANDFORGET);
	// Normally, you would place the conn in a poll and track it's state of course.
	// For example, a simple HTTP server would have a structure for that handle
	// and figure out to do on each event based on what state it's in.
	// Of course, additionally, we'd register/handle all appropriate events like
	// READYREADY, EXCEPTION, etc.
	// Here though we'll just do a simple send-and-hup.
	events_on(wls->events, CONNECTION | WRITEREADY, conn, socketworker_can_write, wls);
	events_on(wls->events, CONNECTION | HUP, conn, socketworker_conn_hup, wls);
};

socketworker_can_write(handle events, handle conn, worker* wls /* this was "userland" elsewhere */) {
	// Still in CPU1..n
	// What's interresting here is that WAITFOR _usually_ doesn't actually block
	// this time because this function is _only_ fired when the handle is writable.
	// We can extend this so it's always non-blocking of course (even when the buffer is full)
	// by simply providing a WRITESIZEREADY event with an exact size for what we want to
	// write.
	write(handle, conn, "Hello there!\nAnd goodbye...", WAITFOR);
	close(handle, HARD | FIREANDFORGET); // immediate if possible, queued if necessary, killed if sys queue limits are reached.
};

socketworker_conn_hup(handle events, handle conn, worker* wls) {
	// Still in CPU1..n
	// Note that the below isn't worker-local
	events_trigger(events, USERLAND, userland_event, "whatever userland param\n" /* << this could be a struct ref or handle or whatever */);
};


arbitrary_userland_event(handle events, handle event, void* inital_bound_arg, char* triggered_arg) {
	// Haven't decided if we're in CPU1..n, or CPU0 (the registrar of the event) here... probably will be flag-controlled.
	// Whatever, now a user-defined event has been fired and we can do whatever with the args.
	// Just an example.
	write(logger, triggered_arg, FIREANDFORGET);
};

after_one_hour(handle events) {
	// Yes, yes, I know, the args aren't right, but it's a prototype example. Simply a unhindered chain-of-thought.
	write(logger, "It's been an hour and I need to go to sleep... good night.\n", WAITFOR);
	process_term();
};

bwat wrote: 1) It's nice to see that you are not wanting to be POSIX/BSD/LIBC/Whatever compliant. However don't be surprised if you implement large chunks of behaviour defined by some standard. My rule of thumb is 75% of an ANSI/BSI/IEEE/ISO standard is the result of hard won experience, 25% is there to protect the economic interests of current implementers and raise the bar of entrance to newcomers. Try to identify the 75% which is good and you'll be getting all that experience on the cheap.

2) As Alan Kay said "The future is not laid out on a track. It is something that we can decide, and to the extent that we do not violate any known laws of the universe, we can probably make it work the way that we want to.". I think the only thing that will stop you with this design is you. Keep on keeping on and you'll get her finished.

mrstobbe wrote: Any immediate technical thoughts on the general idea?

mrstobbe wrote: EDIT: As a name... I meant using "OdinOS" as a name (not as some kind of product that I'm "using")... I quite like it for a bunch of synchronistic reasons and would love to hang onto it if no one's using it.

mrstobbe wrote:Single-process, multi-task. SMP necessary, but 4+ core/cpu highly recommended.
[*]General IRQs firing on CPU0, drivers running on CPU0, any general task/gc on the ring0 level, and the main process on CPU0.
[*]Main process "workers" are sticky to CPU1..n and there's no preemption except during panic.

OSwhatever wrote:you mention that is should be SMP then certain workers are "dedicated" to specific CPUs which contradicts the whole SMP paradigm

OSwhatever wrote:To me this sounds like you are not utilizing the hardware to its full potential. [snip] The point of SMP is to spread the load evenly in order to utilize the processing power as much as possible. Also most interrupt controllers on SMP systems are designed to pick the most appropriate CPU so usually that job is already done for you.

mrstobbe wrote:Oh yeah, one more important point about why I think the mono-process/multitask design with the tasks sticky to each CPU will be more efficient: correct me if I'm wrong, but doesn't a CPU's internal caching (TLB, opcode, and mem caches) and most (if not all???) of it's branch prediction accuracy get thrown out the window every time a context switch happens?

Brendan wrote:That depends. [... snip]

Brendan wrote:Finally; there's a severe difference in magnitudes here. For example, to avoid a task switch that might cause a few microseconds of delays, you're considering wasting entire CPUs for many seconds.

mrstobbe wrote:

Brendan wrote:That depends. [... snip]

All of that makes sense (although, in terms of mem caching, a mem cache is implicitly invalid once a different task starts doing stuff with memory, right?).

mrstobbe wrote:

Brendan wrote:Finally; there's a severe difference in magnitudes here. For example, to avoid a task switch that might cause a few microseconds of delays, you're considering wasting entire CPUs for many seconds.

Wait, what? Explain if you could please. I can't even remotely envision this at all.

Brendan wrote:"Multi-tasking" is running multiple processes at the same time, and doesn't make any sense for a "single-process" OS. I think you mean "single-process with multi-threading and a limit of one thread per CPU".

Brendan wrote:"Much more complicated" and "much more elegant" are opposites. It's probably a bad idea to have an impossibility at the heart of your kernel.

Brendan wrote:The idea of a micro-kernel is to isolate drivers from each other and other processes (and the kernel). With drivers running at CPL=1 or 2 they can trash a process' code and data, and (by loading each others segments) trash each other. What you've described is a monolithic kernel with drivers running at CPL=1 or 2 (and not a micro-kernel at all).

Brendan wrote:Please note that segmentation sucks (it's slow on 32-bit 80x86 and not supported on 64-bit 80x86 or anything else).

Brendon wrote:For critical systems, "single-process" means severe downtime for maintenance (you can't do backups, mount new file systems, update software, etc. in the background, and have to take the critical system offline to use other processes). It also means that you can't have a "keep alive" process (e.g. a process that monitors another process, and restarts the other process if it crashes).

Brendon wrote:For IRQ handling; with all IRQs (and all device drivers) limited to a single CPU you won't be able to handle the load of high speed networking (e.g. two or more gigabit ethernet cards running near max. bandwidth); which will probably make the OS "undesirable" for certain things (e.g. HPC).

Brendon wrote:I think the OS you were thinking about is BareMetal OS.

Brendan wrote:The end result is still cache misses. However, this situation may not happen (e.g. cache large enough to hold data from several tasks) and it may happen without any task switches involved (e.g. cache too small to hold one task's data).

Brendan wrote:Now imagine you also want to run an FTP server too.

mrstobbe wrote:

Brendan wrote:"Multi-tasking" is running multiple processes at the same time, and doesn't make any sense for a "single-process" OS. I think you mean "single-process with multi-threading and a limit of one thread per CPU".

I thought that was obvious in both the description and the details. You were confused about it? Think other people will be too? Should I rewrite everything in the original to call them "threads" instead of "workers" or "jobs"? (serious question... not sarcasm).

mrstobbe wrote:

Brendan wrote:The idea of a micro-kernel is to isolate drivers from each other and other processes (and the kernel). With drivers running at CPL=1 or 2 they can trash a process' code and data, and (by loading each others segments) trash each other. What you've described is a monolithic kernel with drivers running at CPL=1 or 2 (and not a micro-kernel at all).

Well, the idea was to not give them access to each other's memory space, just as they wouldn't have access to the kernel's... you're saying that's not possible? I've never played around with ring1-2. I also was not planing to give them each a segment, but do the standard thing of managing cr3 and rely on paging (similar to ring3 but with slightly higher privileges to do certain things). Worst case I can make them ring3... after all, CPU0's sole job is to be the classic preemptive multitasker and manage stuff.

mrstobbe wrote:The keep alive process thing... why couldn't that be a wrapped launcher for the process (like mysqld_safe?)? It would be sleeping until the process ends and never context switch. When the process ends, it could easily figure out why and react accordingly.

mrstobbe wrote:

Brendan wrote:For IRQ handling; with all IRQs (and all device drivers) limited to a single CPU you won't be able to handle the load of high speed networking (e.g. two or more gigabit ethernet cards running near max. bandwidth); which will probably make the OS "undesirable" for certain things (e.g. HPC).

Hmmm... that could be a problem. Need to think about that more. The whole IRQ management part of this is still fuzzy in my head and I'd like to have a firmer grasp on it by the time the basics are done (I'm still working on simple/generic things like paging and clock management and the like).

mrstobbe wrote:

Brendan wrote:The end result is still cache misses. However, this situation may not happen (e.g. cache large enough to hold data from several tasks) and it may happen without any task switches involved (e.g. cache too small to hold one task's data).

In modern use (depending on use of course), this seems incredibly likely to happen frequently.

mrstobbe wrote:

Brendan wrote:Now imagine you also want to run an FTP server too.

Aaaaahhh... source of the confusion. I stated explicitly that this OS doesn't do more than one major thing. Period. It's only an HTTP server, or it's only an FTP server, or it's only a memcached server, or it's only a [enter something appropriate here] server. The FTP server doesn't seem like a good fit for this type of OS, but a high-volume HTTP server certainly does. Recommendations to help clarify that in the original?

mrstobbe wrote:EDIT: And to clarify a bit more... I understood what you where saying... I'm saying that the point of this OS is to be as idle as possible when idle, and to be as active as possible when active. I don't consider "down-time" to be "wasting seconds". Last time I checked down time was far from a bad thing. The energy consumption alone is a serious concern of most major server environments, and those same environments strive to segment everything exactly as I'm describing... one major purpose per server. At the same time, they want the best they can get under full-load.