OSDev.org

Posted: **Tue May 28, 2019 6:21 am**

Hi All,

Many of you have GUI, but seeing the screenhots topic it seems to me that displaying fonts is usually problematic for most of you. Let me help you with a small and portable font renderer. It was written in a single ANSI C/C++ header file and licensed under MIT, has minimal dependencies, so it's very easy to integrate into your OS. It's API has two modes:

ssfn_putc(unicode)

Is a very simple version, which can display proportional but unscaled bitmap fonts. For this limitation in return it does not allocate memory or call any functions, therefore it is completely dependency free, and compiles into less than a kilobyte. It was designed to be used for OS kernel consoles.

ssfn_render(ctx, unicode)

Is a fully blown version, which needs libc (memset(), memcmp(), realloc() and free()). It can scale glyphs, handles bitmap, pixel and vector based fonts. It's very easy on memory (usually allocates no more than 32 kilobytes), simple to use and compiles about to 21 kilobytes. It was designed to be used by applications, but you can easily integrate it into your kernel if you provide realloc() and free() with some defines. Despite of it's small size, it can do everything that other renderers, and in some respect even more:

UTF-8 support
scaling
antialiasing
automatic hinting
kerning

See the attached image that demonstrates the features.

The font library uses it's own font format I designed, which uses compression on fonts. For vector based fonts you can set the compression level, so you can choose between font quality and file size to your needs. With the same quality as an OpenType, my format usually takes half the file size. Converters (small command line tools also written in ANSI C) are provided to import the following font formats:

PostScript Type 1 (eps)
TrueType (ttf)
OpenType (otf)
X11 Bitmap Distribution Format (bdf)
Linux Console's PC Screen Font (psfu)
GNU unifont (hex)
TARGA (tga, to import pixel fonts)

It comes without absolutely no warranty in the hope that it will be useful.

Cheers,
bzt

Posted: **Tue Jun 04, 2019 9:06 am**

I'm curious about your claim of UTF-8 support. Your API seems to take single code points... how do you handle combining characters?

Posted: **Wed Jun 05, 2019 5:55 am**

Hi,

Solar wrote:I'm curious about your claim of UTF-8 support. Your API seems to take single code points... how do you handle combining characters?

Thank you for checking out!

There's a function, ssfn_utf8() which decodes UTF-8 sequences, adjusts the string pointer to the next multibyte sequence and returns the UNICODE code point (which you can pass to the ssfn_render() and ssfn_putc() functions). For example:

Code: Select all

while(*s)
    ssfn_putc(ssfn_utf8(&s));

There are several ways for combining characters:
1. if UNICODE has specified a combined ligature, then you can use its UTF-8 sequence (mostly in Latin-1 supplement, Extended Latin-A, B and in Alphabetic Presentation Forms blocks, but the latest UNICODE standard has code points for combined Devanagari letters too for example)
2. if combined code point is not specified, then you can get two or more UTF-8 sequences for the combination, all having zero advances except for the last one (as specified by UNICODE in the Combining Diacritical Marks and other blocks)
3. with ssfn_render(), use kerning to zero out the left glyph's advance value
4. with ssfn_render(), ignore the advance values returned, do as you want. This is a low level rendering library, meaning it returns the rasterized glyphs, and you can combine those on screen as you please.

I wrote about ligatures and combination in the file format's documentation, although not much, but here it is:

SSFN can store everything that a TrueType or OpenType font can, except for:

* charset encodings,
* expressed ligatures.

As for character encoding, SSFN exclusively uses UNICODE (ISO-10464) by design. For
ligatures, I suggest to use the proper UNICODE code point for the ligature (which can
be a combined glyph), or a sequence of more code points where only the last glyph has
an advance.

Cheers,
bzt

EDIT: glyph variants for the same UNICODE code point are now supported in SSFN fonts.

Posted: **Wed Jun 05, 2019 6:25 am**

bzt wrote:As for glyph alternatives, SSFN always renders the same glyph for the same
character. So if you want to have different glyphs for a character, you have to use a
different font.

How does this work for things like Arabic text, where the correct glyph can only be determined by the surrounding characters?

Posted: **Wed Jun 05, 2019 9:02 am**

Hi,

Octocontrabass wrote:How does this work for things like Arabic text, where the correct glyph can only be determined by the surrounding characters?

In short, to get the proper glyph you should pass the correct UNICODE code point depending on the surrounding characters.

The long answer is SSFN is a low level glyph rasterizer, and not a high level text layout rendering solution. Just for the records, glyph rasterization does not care about BiDi or Bopomofo composition either. In this regard, SSFN is exactly like freetype2 and not like the Pango library.

Cheers,
bzt

Posted: **Wed Jun 05, 2019 10:45 am**

bzt wrote:In short, to get the proper glyph you should pass the correct UNICODE code point depending on the surrounding characters.

What do you do when there is no such code point, like with Devanagari?

Posted: **Wed Jun 05, 2019 11:26 am**

Octocontrabass wrote:What do you do when there is no such code point, like with Devanagari?

Exactly the same thing that you would do with the freetype2 library.

Again, this is a low level glyph rasterizer, not a text layout rendering engine. Font rendering and text rendering are totally different things. For the latter, see Pango for example.

But to answer your question, your Devanagari text renderer that you implement on top of SSFN, could render the glyph for code point U+000972 if the font has a glyph for that, or use U+000949 in combination of U+000905 to get the same glyph. SSFN is only responsible for returning a rasterized glyph for the given code point (or returns SSFN_ERR_NOGLYPH). Same applies to Arabic texts, it is your text renderer's duty to pass the correct UNICODE to the rasterizer.

Cheers,
bzt

Posted: **Wed Jun 05, 2019 11:59 am**

bzt wrote:But to answer your question, your Devanagari text renderer that you implement on top of SSFN, could render the glyph for code point U+000972 if the font has a glyph for that, or use U+000949 in combination of U+000905 to get the same glyph. SSFN is only responsible for returning a rasterized glyph for the given code point (or returns SSFN_ERR_NOGLYPH).

How do you render a glyph like "द्द" (U+000926, U+00094D, U+000926) which doesn't have its own code point and can't be made by combining glyphs of its constituent code points?

Posted: **Wed Jun 05, 2019 1:17 pm**

Octocontrabass wrote:How do you render a glyph like "द्द" (U+000926, U+00094D, U+000926) which doesn't have its own code point and can't be made by combining glyphs of its constituent code points?

*Sigh* As I have already pointed out, this is a text rendering issue, not a font rendering one.

The UTF-8 text support included with ssfn_utf8() in this simple font rendering library works "only" for all European languages (covers Europe, North-America, South-America, Australia, big part of Africa), all Slavic languages (Eastern-Europe, North-Asia), Chinese, Japanese and Korean languages (majority of Asia, including vertical texts) and most archaic writing systems (ancient Greek, Runic, old Hungarian, etc.). I'm so sorry that this ca. 300 bytes of code does not work for all the complex writing systems that humanity ever think of throughout the history! (imagine the sarcasm emoji here which does not have its own UNICODE code point either, but it really should)

But just to satisfy your curiousity, one way to solve this: you could specify a glyph for "द्द" in the Private Use Area block, and then when your text renderer sees U+000926, U+00094D, U+000926 sequence it will render that single code point instead of these three. Or you can load a font with only special supplementary symbols in it and use that when your text renderer sees special UNICODE combinations (in this case you can put the glyph for "द्द" anywhere, let's say in the place of "A", it's totally up to your text renderer, the font renderer won't care).

Cheers,
bzt

Posted: **Thu Jun 06, 2019 3:09 am**

So, in other words, it can only render text for which there is a direct 1:1 correspondence between glyphs and code points.

Are there any plans for your converters to expose context-sensitive glyphs in some way, so that there's less manual work required for complete Unicode support?

Posted: **Thu Jun 06, 2019 12:44 pm**

Hi,

Octocontrabass wrote:So, in other words, it can only render text for which there is a direct 1:1 correspondence between glyphs and code points.

You should really read the documentation. It does not render text. It renders glyphs, because it is a font renderer, not a text renderer. And there is no direct 1:1 correspondence between glyphs and code points. Never was.

What you have meant I think, is that SSFN's supported encoding is and always will be ISO-10464, and you always get the same glyph for the same character with the same font? Because there was only a guaranteed correspondence between code points and glyphs but not between glyphs and code points. But that's for the past, because since yesterday I've implemented glyph variants, to support Serbian / Macedonian Cyrillic B D G P T letters in Italic style.

But still, within a variant lookup table, you'll get exactly the same glyph for the same code point, but nothing says that the same glyph can't be returned for a different code point too. Actually this is a very important feature that SSFN utilizes for it's geometrical compression algorithm. What do you think, how on earth can it compress a 64k BitStream Vera TTF into a 15k SSFN otherwise?

Octocontrabass wrote:Are there any plans for your converters to expose context-sensitive glyphs in some way, so that there's less manual work required for complete Unicode support?

I'm not planning to change the current behavior, because:

- First, current font formats does not have such information either (there's no place for them in PSFU, BDF, PST1, TTF fonts, and freetype2 does not expose OTF's many different alternative and replacement tables in a single meaningful way, not to mention that neither Uniscript nor Apple renderer can't use those tables properly either, read FontForge's doc)

- Second, it is completely wasteful to store such an information in a font file because a specific UNICODE code point has to be translated to the same corresponding isolated, initial, medial, final UNICODE code points regardless if the font is Serif, Sans, regular, oblique, bitmap, vector based etc. Same stands for UNICODE code point combinations (VS included). If UNICODE hasn't allocated a code point for that letter in a particular context, then you can always use Supplementary and Private Use Areas until they do.

- Third, it's totally absurd and irrational to expect full UNICODE text rendering capabilities from a simple font renderer that's aim is embedability and extra small code size (ca. 20 kilobytes, man!).

- Fourth, why should I bother at all when there are excellent OpenSource libraries to do exactly that? I mean if you want complete UNICODE support, you can use Pango for example. You always forget about Pango.

Cheers,
bzt

Posted: **Fri Jun 07, 2019 3:09 am**

bzt wrote:If UNICODE hasn't allocated a code point for that letter in a particular context, then you can always use Supplementary and Private Use Areas until they do.

Unicode won't assign code points for characters with context-sensitive glyphs unless it's required for compatibility with a legacy code page. There are no such legacy code pages for Devanagari, so Unicode will never assign those code points. That's why I'm asking if there's another way to get those glyphs (that doesn't involve manually assigning them to private use characters).

bzt wrote:why should I bother at all when there are excellent OpenSource libraries to do exactly that?

Why did you bother writing a font renderer when there are excellent open source libraries to do exactly that?

Posted: **Fri Jun 07, 2019 9:37 am**

To be honest, I don't get why people bash this project. bzt never claimed that his library does text shaping (he made it incredibly clear that it doesn't). Hence, it's feature set is really comparable to that of the established competitors (i.e., freetype, which doesn't do shaping either). There is really no reason to combine shaping and rendering, those are separate problems.

I guess one could always slap harfbuzz on top of this to handle shaping.

Posted: **Fri Jun 07, 2019 10:40 am**

Hi,

@Korona: Thanks!

Octocontrabass wrote:That's why I'm asking if there's another way to get those glyphs (that doesn't involve manually assigning them to private use characters).

And I have already told you. Besides of Supplementary and Private Use Area assignments, you can also use glyph variants if you want. But here's the thing: in your font you can assign whatever glyph you want for your text renderer. In the feature demo image above, the stoneage.sfn font for example has a TREE in place of '@' and a VULCAN in place of ';'. Does it work? Yes!

Octocontrabass wrote:Why did you bother writing a font renderer when there are excellent open source libraries to do exactly that?

Let's count: how many hobby OSes are there with a GUI, and how many have integrated freetype2? See?

There aren't really any libraries suitable for hobby OSes. They are huge, bloated, difficult to integrate. Freetype2 shared library on my Linux machine is almost 1M, bigger than my whole kernel, and requires hell lot of a dependencies (bz2, png, z, glib, graphite, thread and even pcre...). FreeSerif in OTF format is over 2M. With SSFN, there's no need for shared library support at all, the whole renderer compiles to about 20K, and FreeSerif in SSFN format requires only about 600K. (And without gzip compression, using only the geometric compression algorithm at level 4 it is still only 1M, half the size of the original).

What's more, it is very problematic to use freetype2 and other existing font renderers for kernel consoles, in contrast to ssfn_putc(), which does not allocate memory, has absolutely no dependencies, and compiles to about 1K of code. True with ssfn_utf8() it can support "only" about 99.9% of the spoken languages and writing systems, but seriously, who wants to read the Koran or the Bhagavad Gita on a kernel console??? That's for displaying string constants (like "starting", "initialized", "loaded", "General Protection Fault" etc.), some register names and abbreviations (like "eax", "ebx", "PCI", "ACPI" that no sane man would ever want to translate), and a bunch of hexadecimal numbers.

Cheers,
bzt

Posted: **Thu Aug 01, 2019 4:55 pm**

Very good initiative.

The collaboration between glyph rendering and text shaping took me a long time to understand, I can understand why some people expected this to do text shaping.

Unfortunately, the rendering quality doesn't look all that good. Do you want to give a brief overview of the algorithm it uses?

OSDev.org

Simple font rendering library

Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library

Re: Simple font rendering library