shared memory and cow

[Solar]

Actually, the x86 supports PIC extremely well. That's exactly the problem segments were designed to solve.

That's irony, yes? Please, please tell me that's irony...

[Candy]

Actually, the x86 supports PIC extremely well. That's exactly the problem segments were designed to solve.

That's irony, yes? Please, please tell me that's irony...

I'd say misunderstanding. Segments solve the memory-availability problem, plus protection. They can also be used to link together multiple non-PIC pieces of code to form an application without the PIC overhead and/or complexity.

PIC is code you can put at any position, not just at the start of a segment.

[Pype.Clicker]

i'd be interrested by some studies that shows the performance impact of shlib implemented as segments compared to the impact of shlibs implemented through fake-PIC-without-relative-addressing ...

does the fact that you have one extra register beat the cost of reloading data segments when changing domain ? does the additionnal references required to read far code pointers exceeds the cost of trampolining ? ...

@candy: please, tell me that AMD64 has a rip-relative addressing mode for data ... please ...

[Candy]

@candy: please, tell me that AMD64 has a rip-relative addressing mode for data ... please ...

*tells you what you request*

Well actually, it really does.

[Brendan]

Hi,

well, problem is that some large program like mozilla use over 100 shared libraries ... and you may expect any component-oriented software to be at least half that big (usually one shlib per class/package )

gnumeric: 50 shlibs.
the gimp (only one snapshot taken): 22 shlibs
xemacs : 38 shlibs
dia (vector drawing) : 58 shlibs
konqueror : 56 shlibs (increasing while browsing new folders)...

Does anyone think this number of shared libraries is sane?

From my point of view a shared library should only ever be created if that library is going to be re-used by at least 3 different programs, otherwise it should be statically linked or part of the programs code. If Mozilla & Konqueror are implemented as single large programs that handle html, java, file browsing, text files, etc then IMHO they should be split into a collection of seperate programs that use IPC to talk to a single "user interface process" (or the GUI directly).

If you wouldn't mind explaining what each of the >100 shared libraries that Mozilla uses are for, and how they can be re-used by other programs...

I'm considering using the "fixed slots" idea for my OS. I'd have one for the kernel, so that user level code wouldn't need to transfer to system level code for kernel functions that don't need CPL=0. I'd have one for the GUI which would provide a common look & feel on top of the primitives provided by the raw IPC GUI interface.

Apart from these 2 chunks of shared code (and the kernel itself I guess) I'm at a loss and can't think of a reason for any more shared libraries. This is most likely due to a gap in my knowledge though - I honestly can't think what people are using them for ???.


Cheers,

Brendan

[Schol-R-LEA]

Actually, the x86 supports PIC extremely well. That's exactly the problem segments were designed to solve.

That's irony, yes? Please, please tell me that's irony...

If you mean, was that sarcasm, well, sadly, no it isn't, at least not entirely. When IBM developed segmentation for the System 360, part of the purpose of it was a way of avoiding the need for PIC, at least relative to the global addressing. Since IBM had not yet adopted virtual memory, and since the original 360 had no hardware stack or indirect addressing (IBM claimed they were too expensive), they devised a segmentation memory system to allow each program to run from a fixed address without conflicting with each other.

Later, when Intel designed the 8086, one of the original goals was source-compatibility with the 8080; even when that proved infeasible, they still wanted to make it easy to port 8080 assembly programs to the new chip. They also wanted to extend the address space, but knew that a 32-bit memory bus would be far to expensive, and would break a lot of the 8080 legacy programs; besides, they figured, very few single programs would ever need more than 64K of memory. Finaly, they figured that, with that huge 1M address space, people would surely want to multitask on the 8086, so they needed some kind of memory protection; however, a memory manager capable of paging and virtual memory was out of the question - it would cost more than the chip did. Segmentation, for all it's flaws, seemed to offer a solution to all of those issues.

I don't know if this is what Dreamsmith was referring to or not, but it certainly seems relevant.

However, if you meant that it is ironic, I would certainly have to agree.

[Pype.Clicker]

Does anyone think this number of shared libraries is sane?

If you wouldn't mind explaining what each of the >100 shared libraries that Mozilla uses are for, and how they can be re-used by other programs...

Well, i can't explain for the 100 of them, but you have several "groups":
* libgtk,libgdk, libX*, /usr/X11R6/lib/* (which is already 12 of them) : used for communication with the GUI, etc.
* a bunch of mozilla-specific shlibs (mailview, cookie, webbrwsr, composer) that are here because mozilla is component-oriented and each component is a shlib in the common framework (e.g. they are sort of "plug-ins" to the main binary file).
* pthread, gmodules, nss_files, nss_dns, esd, audiofile, UTF-16 etc which are there because some feature of the 'standard" libraries are implemented through plug-in shlibs aswell ...

Whether it's "sane" or not ... hmm ... hard to tell. Considering the fact that each of those shlibs have been developped by a different team, having them shlibs improve the probability that they work together (restricted interface and so on). Replacing them all by IPC-connected programs ? that's what kde does in a way (using DCOP server, etc). And you could certainly have the composer and the navigator of mozilla be two separated programs with a common base, but i'm not sure it would reduce the amount of shlibs at all (they share dedicated widgets, theme management, DOM inspector, rendering functions ...)

Now probably that those 100 shlibs could be grouped into a few 'collections' of shlibs that are often seen together (like all the sound things, or the GTK+GDK+X libraries) and that *one* slot could be assigned to each *group* of shlibs, leaving the internal organization an issue to be solved within the group. At least, it would ensure that commonly used shlibs get a greater chance to have their 'favourite' address and that other shlibs can be loaded nonetheless ...

[bkilgore]

Apart from these 2 chunks of shared code (and the kernel itself I guess) I'm at a loss and can't think of a reason for any more shared libraries.

I can think of quite a few reasons...

What about things like a tar library and an ssl library and other such utilities that allow many programs to use a commonly reused set of functionality without have to reprogram it each time, or increase the size of each executable. If you have 20 executables that all need to use 1MB of code that allows them to do the compression they need, it makes more sense to have all the code in one place than adding 1MB to each program.

Also, it is sometimes convenient to compartmentalize functionality to be able to swap in and out functionality depending on the circumstances. For example, a program could be designed to support whatever compression libraries it can find on the system, and then you as the user can install the ones you want it to use, instead of it shipping compiled with all 300 different types the developer thought you might want to use.

These are just a couple of examples, but I certainly see a reason for shared libraries, although I agree that they can add a lot of overhead and create other problems.