Page 1 of 1

unicode-org icu - Super huge icudt72l_dat!

Posted: Thu Oct 06, 2022 3:16 pm
by AndrewAPrice
Has anyone else dealt with porting https://github.com/unicode-org/icu? The stuff inside of icu4c/source/data builds a super huge icudt72l_dat.S, that when statically linked into my applications, causes the binaries to grow by 43MB! :evil:

It's read inside of source/common/udata.c in doLoadFromCommonData and openCommonData via the macro U_ICUDATA_ENTRY_POINT. I noticed the macro ICU_DATA_DIR_WINDOWS, but I'm going to have to dig into it a little more to see if I can move it to a file. And perhaps implement processes sharing the memory of files.

Re: unicode-org icu - Super huge icudt72l_dat!

Posted: Thu Oct 06, 2022 3:19 pm
by nexos
Does your OS support shared libs? If so, then ICU can be a shared lib, and that will reduce the amount of memory / disk space used.

Re: unicode-org icu - Super huge icudt72l_dat!

Posted: Thu Oct 06, 2022 3:48 pm
by Octocontrabass

Re: unicode-org icu - Super huge icudt72l_dat!

Posted: Thu Oct 06, 2022 3:54 pm
by AndrewAPrice
Unfortunately I don't have support for shared libraries. For now, I'd like to load it from a file. I'll change the U_ICUDATA_ENTRY_POINT macro to be a pointer to a function that lazily loads the data file into memory, then returns a pointer to it.

(My microkernel does support shared memory between processes. For large read-only data files, perhaps I could make a service (maybe the VFS) that reads files into shared memory. I'll have to update my shared memory to support one-writer/multiple-readers.)

Re: unicode-org icu - Super huge icudt72l_dat!

Posted: Thu Oct 06, 2022 3:55 pm
by AndrewAPrice

Re: unicode-org icu - Super huge icudt72l_dat!

Posted: Fri Oct 07, 2022 6:16 pm
by AndrewAPrice
I got it to build with the archive file. (I use a custom build system so I'm unravelling what the make rules do and reimplementing it in my system. I had to link in stubdata but then set the macro to the directory where I put the generated .dat file.)

I ported icu because it's a dependency of Skia. I actually have Skia building and drawing my UI widgets now, except text. I have an issue with fontconfig not being able to parse my fonts.conf.

Re: unicode-org icu - Super huge icudt72l_dat!

Posted: Mon Oct 10, 2022 4:38 pm
by AndrewAPrice
I've linked in the stubdata and provided the .dat archive on disk. I'm logging file accesses in my vfs server and noticed it's not being accessed.

My more pressing issue right now is speeding up fontconfig.