shared memory and cow

[Adek336]

Hi folks!
I am thinking about shared memory right now. How do I keep track of the shared data? I might mark such areas as AREA_SHARED and assign a global object to them, telling which spaces are accessing it etc. Should I allow sharing areas which aren't in page granulity, align? And can the shared area have lazy pages, which are not yet allocated? One solution would be to tweak the lazy-page-fault-handler so it would page into all the spaces.

And what about cow? Should this and shared memory be built into the same logic? (the same routines) Then, each memory space which gains access to the data would have a flag whether it wants cow or not. Then, in all the spaces which have the data mapped in, the data would be mapped ro. Any rw access would make a copy of the page for all the spaces which want cow.

It would need some tracking, if a memory space has already a full copy of the object and no longer has a common page with the original object, so I would get rid of the space from the object's list.

Hope this is at least minimally logical...

Cheers,Adrian

[jinksys]

to whoever answers this post:
please explain what cow is, Ive never heard of that before.

[mystran]

to whoever answers this post:
please explain what cow is, Ive never heard of that before.

COW means Copy-On-Write, and the idea is that a single page of physical memory is shared between two (or more) processes, even if they both have a logical copy of it. The page is marked "read-only" in both processes, so that either of them trying to write on it will cause a page fault. When such a page-fault is generated by some process, a copy of the shared page is made, and that copy is given (as read/write) to the fault-causing process. Ofcourse if the last process "sharing" the page causes a fault, the page can be given to it without copying.

Copy-on-write is also used in other context. At least Qt uses copy-on-write strings: the contents of the strings are only copied when on of them is modified. Until then, they all share the same data.

The idea with copy-on-write is that most pages are never written to, so only making private, modifiable copies when they are actually needed saves memory and the time needed the make the copies.

[Solar]

A concept that, unless I am mistaken, is tightly coupled with the concept of fork(), where a whole process is "copied" by using COW. That means you have two copies of the same process running, with minimal overhead unless one of the two copies makes significant write accesses to its address space.

Is there another area where COW does have that much of an impact? I mean, does COW make sense for a system that doesn't employ fork()?

(Sorry for hijacking this thread for that a-bit-OT question.)

As for Adek336: You usually don't keep track of the data at all. Processes sharing memory define their own protocol on what they do in that memory area.

SHM (shared memory) should always be a properly aligned section; whether accesses in that section are aligned or not is, again, up to the processes and their protocol.

Lazy paging... now, that's up to you and how you like your memory handling.

[Legend]

Well, in Linux fork seems (or at least seemed) to serve 2 purpose: Fake threading and spawning new processes.

As you can read here: http://www.die.net/doc/linux/man/man2/execve.2.html
Those exec calls overwrite the current process, so you fork and call that function then. At least it is one way, I don't know if there are others

For threading, the linux scheduler still does not seem to know about threads, so you fork and keep at least the impact on memory low (and the impact on speed was not that hard either, did some benchmarking for work last year, the other OS that it was compared to was however AIX, ugh)

I think if you solve these two issues in other ways, you don't need a fork call, and no copy on write ...

This means a task/thread model for your scheduler and an exec call that does not kill your process.

However the second thing means possibly not being able to comply to SVr4, SVID, X/OPEN, BSD 4.3. POSIX does not document the #! behavior but is otherwise compatible. (quote from the link), so for example this reason, for a *nix you would have to implement fork and copy on write (unless however you can fork without copy on write ).

EDIT: Found some reference to threading in this page: http://www.die.net/doc/linux/man/man2/clone.2.html
However, it seems somewhat related to fork

[Solar]

However the second thing means possibly not being able to comply to SVr4, SVID, X/OPEN, BSD 4.3.

Ah, what a loss...
</sarcasm>

;D

[Legend]

Exactly
Yet another reason to get rid of those!

[bkilgore]

Copy-on-write is also sometimes used in shared libraries. At least in windows, some variables in a dll can be specified as global (not copy on write, synchronization must be used when dealing with these) while others can be marked so that they are copy-on-write and if a process modifies it, it will get its own copy. Basically, COW is useful anytime you have more than one process sharing the exact same data but you want them to behave as if they each have their own copy.

[Legend]

This might involve funny relocations, too ... :-\

[Pype.Clicker]

This might involve funny relocations, too ... :-\

that explains why ELF shared object try so hard to use PICode and forces the use of a Procedure Linkage Table rather than patching straight in the code the external relocations. But that means larger program and more runtime overhead (locking seldom registers, trampolines, indirections, etc.)

On the opposite side, windows DLLs have a "favourite location" and pray the kernel so that it loads them at that location. Lucky programs (i.e. office) that are started first and "know" the favourite location of default DLLs have organized their own DLL so that they can all fit the "default location". Programs that come after these and that cannot know the "default locations" of the whole system may require a private copy of virutally every page, meaning increased memory print and longer loading time.

personnally, i'd advocate for a design where programs that wish to use a given set of shared libraries join a group of programs that have the same requirement (both use network, EyeCandy-Widget-ToolKit and HtmlRendering, for instance), and that the group organizes its libraries in 'matching favourite locations', creating on-disk pre-localized copies of the shlib if the "default favourite" of those shlibs cannot be arranged.

[Solar]

The pain is that x86 doesn't really support PIC well. AmigaOS lived quite well with the notion that any shared library had to be PIC and reentrant (i.e., no globals / statics). Library implementors just had to live with it, which made things much easier for the OS...

[Dreamsmith]

The pain is that x86 doesn't really support PIC well.

Actually, the x86 supports PIC extremely well. That's exactly the problem segments were designed to solve. The problem is, most other processors don't have such a simple and elegant solution for PIC, and thus, pursuing the grail of portability (otherwise known as programming for the lowest common denominator), many compilers refuse to properly support them. If you're upset that your x86 compiler doesn't support PIC painlessly, blame the compiler writers, not the chip -- the chip supports it beautifully...

[Brendan]

Hi,

IMHO DLLs (and PIC) sucks. The problem with DLLs is that it doesn't take long before some idiot (Microsoft) releases an incompatible upgraded version of a DLL that my businesses accounting package relies on, which makes me (and my accountant) fume. Um, I mean that installing a new software may break previously working software.

I think it'd be better to use "DLL slots" at fixed addresses, where a couple "slots" are used for the OS and/or kernel library/s, one for the GUI, and about 8 "user slots". The user slots would hold DLLs that are specific to a single software developer. Each DLL would have a "slot number" in it's header, and a process would only be able to use one DLL per slot.

This way you wouldn?t need PIC at all, and you can define standard/generic (backward compatible) interfaces for all OS, kernel and GUI DLLs. If a software developer messes up a DLL that other software they wrote relies on then it'd make them look bad (but not other unrelated software developers).


Cheers,

Brendan

[Pype.Clicker]

well, problem is that some large program like mozilla use over 100 shared libraries ... and you may expect any component-oriented software to be at least half that big (usually one shlib per class/package )

gnumeric: 50 shlibs.
the gimp (only one snapshot taken): 22 shlibs
xemacs : 38 shlibs
dia (vector drawing) : 58 shlibs
konqueror : 56 shlibs (increasing while browsing new folders)...

[Legend]

And I doubt you would want to maintain those apps in one piece ... look at the linux kernel to see how the project goes out of hands (interestingly that they have not lost their overview on it ...)