Localization / UTF support (Setting up a linear frame buffer)

[glauxosdev]

Hi,

The need to switch between text and graphical modes is not a primary need, because you want to run the console (aka terminal) in graphics mode. This way you can support Unicode and scalable fonts (normal, bold, italic, etc).

Regards,
glauxosdev

[rdos]

The need to switch between text and graphical modes is not a primary need, because you want to run the console (aka terminal) in graphics mode. This way you can support Unicode and scalable fonts (normal, bold, italic, etc).

That wouldn't be compatible with a typical console application that doesn't understand fonts and UTF-8. All you would do is to create a console that is painfully slow (and that lacks a hardware cursor unless you emulate that too).

Besides, unless you do a full hardware acceleration, the scaling built-into most laptops that support multiple resolutions is much faster than anything you can do in a higher resolution with scaling. IOW, if you have a full-screen application that can run in 640x480 resolution, if you run it on a laptop with 1366x768 in it's native resolution, this is going to be painfully slow compared to setting up a 640x480 resolution and running it there without scaling.

[glauxosdev]

Hi,

That wouldn't be compatible with a typical console application that doesn't understand fonts and UTF-8. All you would do is to create a console that is painfully slow (and that lacks a hardware cursor unless you emulate that too).

Painfully slow? Anyway keystrokes are received 30 times a second, so there is time to draw a char.

Also UTF-8 should be provided by the kernel, so there is no need for applications to support it, as the kernel does the work for them.

Regards,
glauxosdev

[Brendan]

Hi,

That wouldn't be compatible with a typical console application that doesn't understand fonts and UTF-8. All you would do is to create a console that is painfully slow (and that lacks a hardware cursor unless you emulate that too).

Painfully slow? Anyway keystrokes are received 30 times a second, so there is time to draw a char.

A sane programmer would scale the fonts once then just use the "already scaled" fonts when characters are drawn. For Unicode (where there's a very large number of characters) and for things like multiple font sizes and bold/italic/underline (where you have multiple versions of the same character) you'd probably want to do this in a lazy way (e.g. only ever generate/scale a character's pixel data if it's not already in your "font data cache"). If it's slow, blame the person that implemented it wrong.

Also note that the hardware cursor is ugly. For almost all cases where you're editing text you want a vertical bar (which the text mode hardware's cursor doesn't support) between characters (also not supported) and not a fat/ugly cube; and you're probably going to want a mouse cursor (which the text mode hardware's cursor doesn't support) in addition to the keyboard cursor, and you want to make sure the mouse cursor and keyboard cursor look quite different to avoid confusion (which isn't really possible with text mode where everything looks like vomit).

Of course text mode itself is ugly. It's designed for a 4:3 aspect ratio and characters look too wide on modern wide screen monitors, even the 9*14 fonts look too big and chunky on larger monitors, there's no anti-aliasing, poor/limited choice of colours, etc. For these reasons, even for "monospaced ASCII only" I still wouldn't want to inflict text mode on unsuspecting victims.


Cheers,

Brendan

[rdos]

Also UTF-8 should be provided by the kernel, so there is no need for applications to support it, as the kernel does the work for them.

Not relevant. An application written for text-mode doesn't expect kernel to interpret texts as UTF-8, it probably wants to draw boxes for the UTF-8 escape characters. In addition to that, it expects all the characters to be of the same width and height, and might also want to use this for changing line automatically. Interpreting text as UTF-8 will break all of this.

[glauxosdev]

Hi,

An application written for text-mode doesn't expect kernel to interpret texts as UTF-8

If an application is written for text mode you can only blame it for not expecting the superior kernel to support UTF-8. On the other side, if an application is written for graphics mode it can simply use the system calls provided by the superior kernel that supports UTF-8.

it probably wants to draw boxes for the UTF-8 escape characters.

7-bit ascii is also valid UTF-8.

In addition to that, it expects all the characters to be of the same width and height

A unicode font can be fixed width (either being a bitmap font, either being a scalable font (check out DejaVu Sans Mono)).

Regards,
glauxosdev

[Octocontrabass]

Interpreting text as UTF-8 will break all of this.

Your application can't display text in several common languages. From my perspective, it was already broken to begin with.

[rdos]

Interpreting text as UTF-8 will break all of this.

Your application can't display text in several common languages. From my perspective, it was already broken to begin with.

A console application (like the command shell), typically is in a common language (like english). It's not for end-users anyway, so that's not an issue. It could also be a deugger-daemon, FTP-server or whatever. None of these are for typical end-users, so there is no reason to support UTF-8 in the text console, let alone that a real text console won't work with UTF-8 anyway.

[rdos]

If an application is written for text mode you can only blame it for not expecting the superior kernel to support UTF-8. On the other side, if an application is written for graphics mode it can simply use the system calls provided by the superior kernel that supports UTF-8.

Yes, that kind of is the essence of it. You don't want an end-user application to used fixed-width fonts, you don't want to use fixed height, "palette colors", or any of that either. Thus, for a professional looking end-user application you want to use a LFB-mode and write text using graphics primitives and all the rest. You don't want to use a text console for that.

OTOH, for an OS utility, like the command shell, you don't want fancy fonts, proportional fonts, underlines, so a text console will work best, and will be faster as you don't need to render fonts.

[glauxosdev]

Hi,

OTOH, for an OS utility, like the command shell, you don't want fancy fonts, proportional fonts, underlines, so a text console will work best, and will be faster as you don't need to render fonts.

And what about input/output filenames encoded as UTF-8? This way the console (as you call it) will be unable to handle them. Linux can handle UTF-8 filenames from the terminal.

Regards,
glauxosdev

[FallenAvatar]

Interpreting text as UTF-8 will break all of this.

Your application can't display text in several common languages. From my perspective, it was already broken to begin with.

A console application (like the command shell), typically is in a common language (like english). It's not for end-users anyway, so that's not an issue. It could also be a deugger-daemon, FTP-server or whatever. None of these are for typical end-users, so there is no reason to support UTF-8 in the text console, let alone that a real text console won't work with UTF-8 anyway.

How does a FTP application that runs in a console that only supports ASCII support UTF-8 filenames and UTF-8 domain names?

- Monk

[Octocontrabass]

A console application (like the command shell), typically is in a common language (like english).

Command shells accept commands in English because the commands need to be universal across all localized versions, not because there is anything limiting the command shell itself to English.

It's not for end-users anyway, so that's not an issue.

System administrators need to be able to access the user's files.

It could also be a deugger-daemon, FTP-server or whatever.

What happens when there's an error message with a non-Latin file name? How do you know which file it refers to?

None of these are for typical end-users, so there is no reason to support UTF-8 in the text console, let alone that a real text console won't work with UTF-8 anyway.

That didn't stop IBM from making DOS/V more than 20 years ago, so why should it stop you today?

[rdos]

How does a FTP application that runs in a console that only supports ASCII support UTF-8 filenames and UTF-8 domain names?

Now your mixing up issues. Whether an FTP application can handle UTF-8 filenames or not doesn't depend on if the console can do it or not. That's an issue if it can write out the filenames correctly or not. If you send UTF-8 filenames to a real textmode console, it will print some funny characters, that's all.

OTOH, I don't support UTF-8 filenames anyway, so the whole issue is non-relevant.

[rdos]

A console application (like the command shell), typically is in a common language (like english).

Command shells accept commands in English because the commands need to be universal across all localized versions, not because there is anything limiting the command shell itself to English.

Exactly. You want the commands to be the same in different languages, and you also want the OUTPUT from the commands to be (because otherwise you cannot parse results programatically). Thus, there is no need for language support in the command shell, and it defeats the whole purpose to have it. Today you can easily expect people that use the console to be knowledgeable in English.

That didn't stop IBM from making DOS/V more than 20 years ago, so why should it stop you today?

Language support in DOS might have been a good idea because typical users actually had to use the shell, but this is no longer true. Typical users no longer use the command shell, they expect something better than that.

[glauxosdev]

Hi,

If you send UTF-8 filenames to a real textmode console, it will print some funny characters, that's all.

So what is the point of having a textmode console if it can't support the most common Unicode standard? This means your system is unusable outside english-speaking countries.

I don't support UTF-8 filenames anyway, so the whole issue is non-relevant.

So it is time to support them. Implementing UTF-8 is not hard, I did it in a single day.

You will never see the beauty of Greek characters in your "console".

Best known ancient philosophers were not english-speaking, so your "console" would be unusable to them, or to others who want to print files containing their quotes.

Keep that in mind,
glauxosdev