OSDev.org

Hi all,

I have been working on my OS for a little bit and have made some progress with some assistance -- thanks to all who have helped. I admit that I am working without a plan or design; my primary objective at this point is to prove to myself that I can understand the mechanics of interacting with the hardware.

So far, I have only produced video output and created a process switching model and have started reading keyboard input. I have been successful at taking keyboard scan codes and echoing them to the screen. It strikes me that I am at a critical design decision point and I thought I would ask for some coaching while I wait for Tanenbaum to arrive.

So far, my keyboard service routine is handling retrieving the scan code from the controller and translating it to a character. I use look-up tables to convert the scan codes to characters, accounting for the different "decorations" for the key such as shift. I queue the character (unicode) and finally return from the ISR. It feels like a performance issue waiting to happen (or happening already and I don't have enough going on to notice).

The scan codes are developed in such a way that I should be able to replace the default key map with a difference one as desired. The thinking behind this is that I set aside unicode f800 - f8ff to be used as special responses and can register handlers for these keystrokes. I expect to use this facility as I continue to build the OS to, say, display the process tables.

I have been pondering simply queuing the scan codes (or messaging them through IPC; not implemented yet) and then waking a process to do the conversion to the unicode key and queuing the result. But that also feels like a possible delay during any period of heavy processing. Or... should the ISR continue to do the translation? I also recognize that though I one option is closer aligned to a micro-kernel and the other is closer to a monolithic kernel.

If I continue down the path of a micro-kernel, it also strikes me that now is the time to standardize on a driver interface (is this called a ioctrl block?). I have searched for ideas on how to approach my questions and have come up with nothing -- I'm certain that I am not using the right terms in my search.

Anyway, the reason for my post, I am asking for some guidance or coaching as I consider my options at this juncture.


Thanks!

Hi,

For most micro-kernels, you'd have IRQ handlers that only send message/s to drivers. The drivers (running in user-space) would receive the "IRQ occured" message and handle it. Because the device driver's message handler is running like a normal process in user-space (and not running as an IRQ handler in kernel-space) a lot of the problems of "doing too much" (e.g. IRQ handlers that take too long and cause latency problems) disappear.

For most monolithic kernels, you'd do what you've done (handle the IRQ entirely in the IRQ handler in kernel space) if the work that needs to be done is relatively small, or you'd split the work into 2 parts - e.g. one part ("top half") that runs in the IRQ handler itself which puts something in a queue or buffer, then another part ("bottom half") that does the rest of the work for handling the IRQ outside of the IRQ handler itself.

eryjus wrote:If I continue down the path of a micro-kernel, it also strikes me that now is the time to standardize on a driver interface (is this called a ioctrl block?).

Some OS attempt to pretend that everything is a file; then they realise that everything isn't a file and have "character devices" (that pretend to be streams) and "block devices" (that pretend to be files); then they realise that it still isn't enough and use "IOCTLs" on top of that as a way to work around all the problems. For a simple example, if you're pretending that a CD-ROM is the file "/dev/cdrom" and using the normal "open()", "read()", "close()" file IO functions, there's no way to eject the CD, so you need something else on top of that.

Some OSs don't try to pretend that everything is a file (or that everything is either a stream or a file) and therefore don't have the "Ooops, I need IOCTLs" problem in the first place. Instead, they might have different interfaces for different types of devices, or try to pretend that "everything is an object", etc. My way is to define protocols on top of messaging/IPC.


Cheers,

Brendan

Brendan wrote:For most micro-kernels, you'd have IRQ handlers that only send message/s to drivers. The drivers (running in user-space) would receive the "IRQ occured" message and handle it. Because the device driver's message handler is running like a normal process in user-space (and not running as an IRQ handler in kernel-space) a lot of the problems of "doing too much" (e.g. IRQ handlers that take too long and cause latency problems) disappear.

For most monolithic kernels, you'd do what you've done (handle the IRQ entirely in the IRQ handler in kernel space) if the work that needs to be done is relatively small, or you'd split the work into 2 parts - e.g. one part ("top half") that runs in the IRQ handler itself which puts something in a queue or buffer, then another part ("bottom half") that does the rest of the work for handling the IRQ outside of the IRQ handler itself.

I have a mixed concept. My IRQ handlers are like in a micro-kernel, but the real processing is not done in a user-space driver, but in a kernel-space thread in a kernel-space driver.

The top-bottom half concept is really ugly. While the bottom half doesn't really run in the IRQ-handler itself, it doesn't run in the context of a handler thread either, so represents some kind of intermediate state. It is not a thread, and not an IRQ, and it is totally impossible to debug with non-intrusive methods.

Brendan wrote:Some OS attempt to pretend that everything is a file; then they realise that everything isn't a file and have "character devices" (that pretend to be streams) and "block devices" (that pretend to be files); then they realise that it still isn't enough and use "IOCTLs" on top of that as a way to work around all the problems. For a simple example, if you're pretending that a CD-ROM is the file "/dev/cdrom" and using the normal "open()", "read()", "close()" file IO functions, there's no way to eject the CD, so you need something else on top of that.

Some OSs don't try to pretend that everything is a file (or that everything is either a stream or a file) and therefore don't have the "Ooops, I need IOCTLs" problem in the first place. Instead, they might have different interfaces for different types of devices, or try to pretend that "everything is an object", etc. My way is to define protocols on top of messaging/IPC.

My way is to implement syscalls per-function, and when appropriate, "everything is an object" concept with handles (handle == object, but with no direct access). Then I implement the "everything is a file" concept in a user-space library when/if required.

Brendan,

Brendan wrote:My way is to define protocols on top of messaging/IPC.

Can you please help me understand this statement? Do you implement 2-day messaging? Or, are you passing predefined or dynamic structures as messages?

I do apologize if this is a ridiculous question.


Thanks!
Adam

Hi,

eryjus wrote:

Brendan wrote:My way is to define protocols on top of messaging/IPC.

Can you please help me understand this statement? Do you implement 2-day messaging? Or, are you passing predefined or dynamic structures as messages?

I don't know what "2-day messaging" is (and google failed to find anything that looked relevant).

For IPC, I use asynchronous variable sized messages. Any thread can send a message containing anything they like to any thread (as long as they know the address), and the sender never has to wait for a reply. The OS delivers the message to the receiver's message queue. The receiving thread checks its queue whenever it wants, and handles the message (which can include sending one or more reply message back). The kernel itself doesn't know or care what a message contains (it just delivers messages, and handles things like unblocking threads that have blocked until a message arrives).

For threads to be able to understand each other, they need to agree on some sort of protocol. The protocol defines which "message ID" is used for what, and the format of any data in each type of message.

For an example, let's say we want to design a protocol to use for a font engine. You might start with something to select a font:
Then you might want to be able to tell the font server that you're finished using it (so it can unload font data, etc):
Next, something to convert a Unicode string into an "alpha channel" bitmap:
Of course for a real font engine, you'd probably want something to get the font engines capabilities (if it supports various optional features), get a list of available fonts, maybe some way of installing new fonts, etc. The "font engine protocol" above is fairly, um, basic..

Once the "font engine" protocol has been defined (and hopefully published as a formal specification); 20 different people can write 20 different font engines that all use the same protocol; and 2000 different people can write 2000 different applications that use any of these font engines.

For device drivers it's very similar - for each type of device you define a protocol. You might start with a generic "file IO" protocol that defines messages for opening, seeking, reading, writing, etc. Then you might use that as the basis for a "storage device" protocol by reusing (inheriting?) the same message types that were defined for file IO, and then adding more message types for ejecting the media, formatting the media, doing "secure erase", etc. Of course you're not restricted to this - for example, the protocol you design for audio devices might be radically different (e.g. you might not have open, seek, read, write because they don't make much sense; and you might have something like "play <filename> at <when> using <volume>" instead).


Cheers,

Brendan

Brendan wrote:I don't know what "2-day messaging" is (and google failed to find anything that looked relevant).

Typed it wrong and proofed it wrong -- Sorry!

I meant to type "2-way messaging", which you answered. Thank you both for your kind replies.