Localization / UTF support (Setting up a linear frame buffer)

[Combuster]

I wonder how it is possible to even have a "painfully slow" console.

By doing this on an USB photo frame

[rdos]

So it is time to support them. Implementing UTF-8 is not hard, I did it in a single day.

I have UTF-8 support in the GUI library. I just don't think the console is a specialized "GUI app".

You will never see the beauty of Greek characters in your "console".

I can in the GUI.

[glauxosdev]

Hi,

I have UTF-8 support in the GUI library. I just don't think the console is a specialized "GUI app".

No, it isn't. But I have to tell you for nth time that a "console" has to have UTF-8 support (or any other Unicode encoding for that matter). Really, just stop.

Ἓν οἶδα, ὅτι οὐδὲν οἶδα.

It means "All I know is that I know nothing". Adopt this quote to your own life and stop thinking you are the superior one here.

Also, save this quote as a file and show it in the "console" at start-up. Maybe you will learn someday...

Regards,
glauxosdev

[gerryg400]

No, it isn't. But I have to tell you for nth time that a "console" has to have UTF-8 support (or any other Unicode encoding for that matter). Really, just stop.

You should stop. Keep your opinion about what someone else's console should do and should not do to yourself.

[Octocontrabass]

OTOH, I don't support UTF-8 filenames anyway, so the whole issue is non-relevant.

So your GUI can be fully localized, except file names, which must be written in Latin characters even if the local language is something like Arabic, which has no standard representation in Latin characters.

You want the commands to be the same in different languages, and you also want the OUTPUT from the commands to be (because otherwise you cannot parse results programatically).

You want the error messages to be in the user's language. You should not be parsing error messages programmatically; that's what return values are for.

Today you can easily expect people that use the console to be knowledgeable in English.

I'm not so sure about that.

Typical users no longer use the command shell, they expect something better than that.

And I'm sure your GUI that forces all file names to be written in Latin characters will be very popular.

[glauxosdev]

Hi,

I agree it leads nowhere...

Regards,
glauxosdev

[Candy]

Split off this topic for the localization discussion.

My view is that we shouldn't localize anything, but do allow the user-generated content to be user-localized. Your app uses standard names for things and it's the same for everyone (so you can google them and so on), but the stuff you make can use unicode names and so on.

[Brendan]

Hi,

My view is that we shouldn't localize anything, but do allow the user-generated content to be user-localized. Your app uses standard names for things and it's the same for everyone (so you can google them and so on), but the stuff you make can use unicode names and so on.

My view is that GUI has dominated since the 1990s; studies have shown that things like icons and WYSIWYG and "discoverable" user interfaces are far easier for users; the ability to script GUI apps has been around for at least 20 years now (Applescript, 1993); and I expect we've all recognised a trend towards mobile devices that don't have a keyboard (and use touch instead, where entering text is clumsy and GUI is far more suited). With these things in mind; a truly modern OS (e.g. where backward compatibility with ancient/obsolete things like CP/M and Unix aren't important) simply shouldn't have a console in the first place.

Therefore; when implementing a console the primary goal should be to minimise wasting developer time on something that ideally shouldn't exist. That would imply recycling the same "text output" code that GUI apps use to avoid wasting developer time on things like supporting text mode and having special "ASCII only" text processing (in addition to the "Unicode text processing" that you have for GUI text boxes, etc); especially when you consider "console running in a window" where it's all going through the GUI anyway. Most (all?) GUIs support full screen applications, and "full screen console" really doesn't deserve to be a special case.

Basically; console should support Unicode, not because it's important to support Unicode, but because OS developers have better thing to do than waste time implementing unnecessary "ASCII only" support.


Cheers,

Brendan

[Antti]

If you send UTF-8 filenames to a real textmode console, it will print some funny characters, that's all.

It might be a valid design decision but you could still handle UTF-8 strings but not render the non-ASCII characters. A band name Motörhead could be encoded as a byte sequence (in hex): 4D 6F 74 C3 B6 72 68 65 61 64.

You could print this in three ways:

• Motörhead. Correct way.
• Mot?rhead. UTF-8 supported but not rendered. Acceptable.
• Motörhead. UTF-8 not supported at all and the string length is "undefined". Worst.

[Candy]

console should support Unicode, not because it's important to support Unicode, but because OS developers have better thing to do than waste time implementing unnecessary "ASCII only" support.

Yes.

I've named my new OS attempt Rødvin so as to force myself to get it right everywhere. If you cannot do unicode somewhere you can't even print the OS name.

Incidentally, it has a console as part of its graphics mode stuff - because I didn't want to waste time implementing ascii-only text mode.

Having everything the same encoding and converting at the door means all internal logic is much simpler.

[rdos]

Basically; console should support Unicode, not because it's important to support Unicode, but because OS developers have better thing to do than waste time implementing unnecessary "ASCII only" support.

Sure, but I didn't waste any time when I did this maybe 20 to 25 years ago, when the available video-modes were really crappy and I didn't even have a GUI because of it. If I did it today, I might have wasted time, but not back then.

[Combuster]

The biggest problem comes when you have files existing on a medium, and you can't do anything with them because you can't enter its filename in any way.

[bigbob]

The biggest problem comes when you have files existing on a medium, and you can't do anything with them because you can't enter its filename in any way.

Perhaps it's not an elegant solution but my Non-Unicode OS can read files with unicode chars from Fat32 on pendrives simply by skipping the unicode chars that are greater than 127 during comparisons. When I print the filenames I print an 'x' instead of (non-ascii) unicode chars. This way I can cd directories-with-unicode-names too.
Similarly when I need to enter a filename in my OS that has unicode chars on the USB-Fat32 filesystem, I print an 'x' at the position of the unicode char (but it will be skipped, so it could be any char). For example: Kxbenhavn (where x is that crossed 'o').
The OS is simple this way.

[alexfru]

my Non-Unicode OS can read files with unicode chars from Fat32 on pendrives simply by skipping the unicode chars that are greater than 127 during comparisons. When I print the filenames I print an 'x' instead of (non-ascii) unicode chars. This way I can cd directories-with-unicode-names too.

Is there any resolution for possible ambiguity? E.g. what if you have files named filé and filè, how do you distinguish fil from fil or filx from filx?

[bigbob]

my Non-Unicode OS can read files with unicode chars from Fat32 on pendrives simply by skipping the unicode chars that are greater than 127 during comparisons. When I print the filenames I print an 'x' instead of (non-ascii) unicode chars. This way I can cd directories-with-unicode-names too.

Is there any resolution for possible ambiguity? E.g. what if you have files named filé and filè, how do you distinguish fil from fil or filx from filx?

Well, that's a problem. The first file will be found in your case.
Have no filenames like those ones
I think it's worth it because of the simplicity of the OS.
EDIT: the length of the filename matters too, so only filX and filY are ambigous, where X and Y are unicode chars.