How into implementing a proper keyboard driver?

[Bonfra]

I do already have a minimal keyboard driver: it catches interrupts, translates raw scancodes to a fixed table of keys (thus loading a keyboard layout that maps scancodes to keys), and allows to either check if a specific key is currently pressed or to poll from a queue of keypresses. That's all good but I'm now facing the problem of the actual keyboard layout: knowing that 0x29 maps to KEY_BACKTICK is not enough, it is also needed to know that KEY_BACKTICK + KEY_SHIFT = VK_TILDE. Now here comes the problem. If I implement this logic at the keyboard driver level I will not be able to catch a CTRL+C keypress because according to the currently loaded layout KEY_C means nothing together with KEY_CTRL. I thought about making the keyboard driver output just basic keys so making the final application (like a tty driver) responsible for translating things to a virtual layout; this will allow to catch raw key combinations and to translate to some printable char if none is matched. This is a viable solution but to reduce the boilerplate code that would form on each application I should create a library that exposes a function like key_to_string(char* buff, uint16 key, uint8 mods). This function would rely on a layout loaded prior to any call, but this layout would be different from the one loaded in the original keyboard driver, so in the end, I need to load two layouts to use the keyboard.
This is really clunky design I come up in a long night and I think something really better can be done. Do you know any technique I can use?

[rdos]

I think the trick is that there needs to be two different interfaces to the keyboard. One is the "raw" interface where you get presses & releases + which control keys are pressed. This interface is used to implement GUIs. The other is to simulate somethig that works like a command line where you only get translated keys. This interface is used to implement tty or command line. Another possibility is to put a translation layer between the raw interface and "tty" in the kernel or in a user level library.

[Bonfra]

So it appears that the existence of two distinct keyboard interfaces, hence two places to load a layout, is needed to have a functional system. Whether the second system is embedded in the final piece of software that needs cooked keys (e.g. text editor) or is provided as an API by some library. In general, the keyboard driver (and so the kernel) needs to expose just raw keys. Idk it seems like some bug-abusable design by the end user to need to load two different layouts; yes a layout file could contain both layouts but it still looks like an overcomplicated solution.

[Demindiro]

I've also been wondering how to best deal with keyboards and the plethora of scancodes.
This problem can IMO best be split up in multiple subproblems:

• Translate arbitrary scancodes to OS-wide consistent "raw" keycodes.
• Apply modifiers to raw keycodes.
• Convert modified keycodes to a stream of UTF-8 characters

Keycode & event format

I first defined a format for the keycodes:

• To ensure I cover all possible characters I use the Unicode character set. This required at least 21 bits (0 to 0x10ffff)
• For keys that can't be mapped to a Unicode character or otherwise need to be distinguishable like Shift, keypad numbers ... I use the 0x110000 to 0x1fffff space that is unused by Unicode.

To distinguish between a press and release event I added another bit.
This allows encoding each keypress as 22 bits. I rounded this up to 32 bits.

Scancode to raw keycode

We then need to map scancodes to keycodes.
For PS/2 I first translate the scancodes to USB HID codes so I don't need to make two keymaps for each keyboard.
For translating the HID (or any) codes I use a configuration file:

Code: Select all

; Based on Logitech K120, AZERTY layout

(raw
	; ordered row-wise, top to bottom, left to right

	(² 35)
	(& 1e)
	(é 1f)
	('"' 20)
	("'" 21)
	("(" 22)
	(§ 23)
	(è 24)
	(! 25)
	(ç 26)
	(à 27)
	(")" 2d)
	(- 2e)
	(backspace 2a)

	(tab 2b)
	(a 14)
	(z 1a)
	(e 08)
	(r 15)
	(t 17)
	(y 1c)
	...

Raw keycode to modified keycode

For modified keycodes I add a section for each modifier combination and a mapping from raw keycodes to modified keycodes:

Code: Select all

(altgr
	(¬ ²)
	(| &)
	(@ é)
	("#" '"')
	(¼ "'")
	(½ "(")
	(^ §)
	({ è)
	...

(caps
	(³ ²)
	(1 &)
	(2 é)
	(3 '"')
	(4 "'")
	(5 "(")
	(6 §)
	(7 è)
	(8 !)
	...

(altgr+caps
	(¡ &)
	(⅛ é)
	(£ '"')
	($ "'")
	(⅜ "(")
	(⅝ §)
	...
	(˙ =))

(Note that scancodes are supposed to map one-to-one to a raw keycode, so using raw keycodes here is fine).

I hardcoded some translations such as a -> A since I don't expect any sane keyboard to put those on different keys.

Modified keycodes to UTF-8 stream

With the modified keycodes this is trivial to do: check for press events with modified keycodes, see if it is Unicode and if yes, just write it out. Otherwise ignore.

[Octocontrabass]

I hardcoded some translations such as a -> A since I don't expect any sane keyboard to put those on different keys.

I guess Turkish keyboard layouts are not sane. (ı -> I and i -> İ)

[Demindiro]

I hardcoded some translations such as a -> A since I don't expect any sane keyboard to put those on different keys.

I guess Turkish keyboard layouts are not sane. (ı -> I and i -> İ)

There goes another one of my assumptions

I guess I'll keep the hardcoded translations but allow them to be overridden.

[nullplan]

Keyboard layouts are a difficult problem, especially if you only know a few, so I would simply try to use the Xkb database. The format there is something like

Code: Select all

keycode 29 = z Z z Z leftarrow yen leftarrow

The first is without modifiers, the second with shift, the third and fourth I don't know (I cannot find any mapping where they don't equal the first and second), the fifth is with AltGr, the sixth with AltGr + Shift, the seventh and eighth once more I don't know. Maybe you need keys I don't have.

Anyway, this format is reasonably good at turning keycodes into keycaps, so you can probably rely on it. Although there is an enormous number of keycaps.

[xeyes]

looks like an overcomplicated solution.

This problem itself is very complicated.

I will not be able to catch a CTRL+C keypress because according to the currently loaded layout KEY_C means nothing together with KEY_CTRL.

This part of the problem is not complicated, there're only a few modifier keys. CTRL + C means something, but KEY_A + KEY_C doesn't.

I'm using a method similar to what has been mentioned a few times, a hand-coded and hard-coded look up table indexed by scan code.

Mine has 4 main cols: normal (1 or a) shift (! or A) caps (1 or A) ctrl (\x82 or \x1), and other aux cols to help with processing. You can customize it in ways that suite your needs and your processing code.

When you get a new keyboard (layout) you'd just need to create another table for it, when you need to support a new modifier key or some other properties for special processing just add a column to it.

piece of software that needs cooked keys (e.g. text editor) or is provided as an API by some library.

IMO this is the diffcult part. If you are interested in porting POSIX apps, you should take a look at the termios interface and think about how would you provide it.

But it also has far reaching implications (tied to job control, as in, how does Ctrl+C propagate within your system), so if you don't want to write another unix clone, maybe try to design your own library and port some text editors before looking too closely at it?

[Bonfra]

Yea I underestimated the problem, now I full-on agree with you: this is hard. Thanks a lot for the suggestion, I'm going to strictly follow the lookup table solution as it seems the more portable one for different (even custom layouts). I'm thinking about something like this

Code: Select all

typedef struct kb_layout
{
    uint16_t physical[UINT16_MAX];
    struct
    {
        uint16_t vk_normal;
        uint16_t vk_shift;
        uint16_t vk_altgr;
        uint16_t vk_shift_altgr;
    } virtual[UINT16_MAX];
} kb_layout_t;

so I can implement a layout like this

Code: Select all

const kb_layout_t kb_layout_en_us = {
    .physical = {
        [0x00 ... UINT16_MAX - 1] = 0xFFFF,

        [0x29] = KEY_BACKTICK,
        [0x02] = KEY_1,
    }
    .virtual = {
        [KEY_BACKTICK] = { .vk_normal = VK_BACKTICK, .vk_shift = VK_TILDE, .vk_altgr = VK_NONE, .vk_shift_altgr = VK_NONE },
        [KEY_1] = { .vk_normal = VK_1, .vk_shift = VK_EXCLAMATION_MARK, .vk_altgr = VK_NONE, .vk_shift_altgr = VK_NONE },
    }
}

And then add a stand-alone method that takes a physical key and outputs a virtual key based on the current modifiers.
Thanks again for all your help

[eekee]

I've concluded I can't do better than X11. For each keypress, it can supply an untranslated keycode, whichever modifiers are pressed/active, and the translated character if there is one.

Keyboard layouts are a difficult problem, especially if you only know a few, so I would simply try to use the Xkb database. The format there is something like

Code: Select all

keycode 29 = z Z z Z leftarrow yen leftarrow

If I remember right, one of these columns is caps-lock; it's treated as a separate modifier. This allows caps-lock to only affect the alphabetical keys. It also allows for all sorts of language-specific peculiarities, such as German having no capital letter eszett.

[Demindiro]

After wondering how to best support keyboards with analog input, where the keyboard can tell how deep a user is pressing a key, I figured I should use the remaining 11 bits to send a signed integer instead of just a single bit indicating whether a key is pressed or not.

Incidentally, this makes it practical to treat the mouse as a very special kind of keyboard: when the mouse moves, send a Mouse{X,Y} event (where Mouse{X,Y} is a keycode) with as press level how much the mouse moved, then immediately send another event that sends the same event but with 0 press level.

This should also make it easier to support joy sticks, since you can send the same Mouse{X,Y} event to move the cursor without adding special support.

[Ringding]

such as German having no capital letter eszett.

This changed a few years ago, though (2017). But of course it does not have a glyph in the VGA BIOS font.

https://de.wikipedia.org/wiki/Gro%C3%9Fes_%C3%9F
https://en.wikipedia.org/wiki/%C3%9F#De ... pital_form

[eekee]

After wondering how to best support keyboards with analog input, where the keyboard can tell how deep a user is pressing a key, I figured I should use the remaining 11 bits to send a signed integer instead of just a single bit indicating whether a key is pressed or not.

Interesting keyboards!

Incidentally, this makes it practical to treat the mouse as a very special kind of keyboard: when the mouse moves, send a Mouse{X,Y} event (where Mouse{X,Y} is a keycode) with as press level how much the mouse moved, then immediately send another event that sends the same event but with 0 press level.

This should also make it easier to support joy sticks, since you can send the same Mouse{X,Y} event to move the cursor without adding special support.

I've always been in favour of treating the mouse buttons as keys, but this is new. Since keys are single-axis, am I right in thinking the X and Y axes would be sent as separate keycodes? I think that would work all right.

What would you do to report absolute mouse position? Oh I know: you could use another pair of keycodes if the API sends axis data as 32-bit integers.

[Demindiro]

I've always been in favour of treating the mouse buttons as keys, but this is new. Since keys are single-axis, am I right in thinking the X and Y axes would be sent as separate keycodes? I think that would work all right.

It does indeed work fine. Technically it's possible for a mouse event to be "torn" (i.e. X axis is processed in one frame and Y axis in the next) but this shouldn't be a problem unless a massive amount of keycodes is sent at once.

What would you do to report absolute mouse position? Oh I know: you could use another pair of keycodes if the API sends axis data as 32-bit integers.

I've considered it but decided against it because a device that reports an absolute position is usually a tablet or a touchscreen, which can register multiple touches and also measure the pressure of each touch. While it would be possible to hack something together with keycodes I think it'll be much cleaner to have a separate API for such devices.

[Gigasoft]

The way I did it, I have applications receiving both key press/release events (which are the same regardless of layout) and character input as separate message types. Keyboard drivers know nothing of characters or keyboard layouts. Instead, a key mapper is automatically created for each keyboard and invoked by the UI to generate character messages. Now, I hope that there aren't languages which put things like backspace, arrows keys and the like at different positions or I'm going to have a headache.

Mice and touchpads are registered with the UI in the same way as keyboards, except that they don't have an associated key mapper. They report input with a separate set of codes (movement, scrolling, button states for up to 5 buttons, and absolute position). X and Y axes are reported together in a single input packet, as they of course should be.

For any other type of input (key pressure, joy sticks, etc), applications should probably connect to the device using a separate API capable of receiving any kind of input report and describing what is in them. Joy sticks certainly shouldn't cause cursor movement.

Incidentally, this makes it practical to treat the mouse as a very special kind of keyboard: when the mouse moves, send a Mouse{X,Y} event (where Mouse{X,Y} is a keycode) with as press level how much the mouse moved, then immediately send another event that sends the same event but with 0 press level.

This should also make it easier to support joy sticks, since you can send the same Mouse{X,Y} event to move the cursor without adding special support.

But why? What benefits do you get from treating a mouse axis like key pressure? When you see yourself forcing some thing to be some entirely different kind of thing it is usually a sign of something needing to be reworked, especially when you end up with things like separate events for X and Y axis movements. Mouse inputs aren't keys, a mouse is used to point at and interact with things on the screen. Joy sticks again have an entirely different purpose, you don't point at things with a joy stick. Joy sticks are usually found on game pads where there are often several of them, and are used for specialized applications such as games. Axis values mean nothing in themselves without knowing what they represent.

I've concluded I can't do better than X11. For each keypress, it can supply an untranslated keycode, whichever modifiers are pressed/active, and the translated character if there is one.

With the traditional processing of PC keyboard input, a single keypress can generate multiple characters if an accent key is pressed followed by another key for which no corresponding accented character exists. For example, ~ followed by G outputs ~g. I recommend keeping to this tradition. Linux forces me to press space in between, or the ~ just doesn't register, which is always a frustration when being used to Windows.