Elf DLL Loading

[prajwal]

Dear All,
I have Implemented an Elf Loader Which is Capable of Loading an ELf EXE into Mem and Excute it. If this Elf exe is dependent on some DLL, the Dynamic Link Loader is able to Resovle the Relocation Address and
a) Load the DLL into Mem If not already loaded
b) Map DLL into particular Process's Address Space.

If one more EXE is Run which also refers above DLL, then Only Mapping (step b) is Done since it already resident in Mem.

So far so fine,

But what if a DLL (Shared Object) is dependent on One more DLL. Each DLL will have its own GOT and PLT.
If for ex,
EXE1 depeneds on DLL1
DLL1 refers to Variable X defined in DLL2,
While executing EXE1, when X is accessed, Dynamic Loader will Relocate and Update GOT of DLL1 with address of X in DLL2. But, DLL1 can be used by a Different EXE2(say) which will have mapped DLL1 and DLL2 to its own Address Space and the Relocated Addrress in GOT of DLL1 for X may not be correct for EXE2.....

Sorry for Such a long explanation of problem... Please help me understanding how Relocation is Done in DLLs dependend on one more DLL

[prajwal]

Ok.. forget about previous Question.

Will there be always One Copy of Shared Libratry in Mem so that Multiple processes access the same Shared Library Mem.

If Yes what about Shared Variables.... in Shared Libray....

Now i have implemented ELF Exe + Shared Library Loading,
Each Shared DLL will be loaded and linked to each process seperately.

Does any one have an idea how it is done in other OSes like Linux, Windows etc...

[carbonBased]

Unfortunately, this isn't an answer to your question, but yet another question -- I wonder if you could post some links to resources you've used to develop your ELF linker/loader?

I also intend to write an ELF linker/loader module in my OS (I currently have a simple RDOFF2 linker/loader at the moment) and would like to find useable docs to implement the same using ELF.

Thanks,
Jeff

[prajwal]

So far, this is the only ELF Document i have referred....

www.skyfree.org/linux/references/ELF_Format.pdf

If u know some more useful links, please share...

[JAAman]

Will there be always One Copy of Shared Libratry in Mem so that Multiple processes access the same Shared Library Mem.

afaik:

in theory yes, but in practice, (at least in windows) it sometimes requires multiple copies, because the virtual address ocupied by shared library is unavailible in one (or more) processes (the shared lib is mapped into local address space, but the patched lib needs to be at the same address in all processes) -- usually because another shared lib is loaded into that same address before that one is referenced (meaning the second lib will need to be at a different address -- if its already in memory at that address, a new copy will be needed patched to a different address)

[prajwal]

If there is only one copy of Shared library in memory, how Global variables in Shared library are delt with.... I mean if one process modifies the Shared Variable that will be reflected in another process... is It?.... Hows this going to work.... any idea....

[Da_Maestro]

it sometimes requires multiple copies, because the virtual address ocupied by shared library is unavailible in one (or more) processes

Are you reffering to when there is one copy in memory and one on the disk? I've heard some of the nut-jobs at Redmond talk about that in the wrong context before.

[Daedalus]

it sometimes requires multiple copies, because the virtual address ocupied by shared library is unavailible in one (or more) processes

Are you reffering to when there is one copy in memory and one on the disk? I've heard some of the nut-jobs at Redmond talk about that in the wrong context before.

I think he's referring to multiple copies in the memory.
Think of it this way, if you were calling a function in the DLL, and a task switch happened at the same time and another process called a function that uses the same memory as what the first process is calling, you'd end up with different processes overwriting the same memory.

Making another copy of the function/functions allows you to have multiple processes using those functions without risk of overwriting itself.

[carbonBased]

it sometimes requires multiple copies, because the virtual address ocupied by shared library is unavailible in one (or more) processes

Are you reffering to when there is one copy in memory and one on the disk? I've heard some of the nut-jobs at Redmond talk about that in the wrong context before.

I think he's referring to multiple copies in the memory.
Think of it this way, if you were calling a function in the DLL, and a task switch happened at the same time and another process called a function that uses the same memory as what the first process is calling, you'd end up with different processes overwriting the same memory.

Making another copy of the function/functions allows you to have multiple processes using those functions without risk of overwriting itself.

Or you could just rely on the library to be smart, and use mutexes where appropriate... thus saving memory, and allowing the same copy to be mapped into every address space.

Global variables are bad...

--Jeff

[JAAman]

im talking about runtime linking -- when an app loads a shared lib to address 0x0123 4567, and another app loads the same shared lib to address 0x0234 5678

you need 2 copies in memory -- one patched to run at address 0x0123 4567 and another patched to run at 0x0234 5678 (which, iirc, is what windows does)

alternatively, you could use PIC code (difficult in PMode), or segments (not possible in LMode, and not permitted in my OS -- and many others, as well as being difficult to implement in many languages -- including C (few compilers support FAR calls))

[prajwal]

It is possible to Load only one copy of dll into memory and map it into multiple user processes address space at a common Address.. Say for all user processes, The Address starting from 2GB is for DLL and a given DLL has to be mapped to the Same Address by all processes....

Advantage is that, if one Process Loads the DLL and Relocates certain dynamic symbols... that need not be redone by any other process which uses that dll in future.....

The only problem here would be with global variables in DLL..... Should DLLs support Global Variables....???

Furthur, since local variables in DLL will be on the corresponding User Process Stack, it shouldn't create problem....

But right now i have gone ahead with approach what JAAman mentioned. i,e,
Each User Process has its own copy of DLL Loaded into its own Address Space

[JAAman]

But right now i have gone ahead with approach what JAAman mentioned. i,e,
Each User Process has its own copy of DLL Loaded into its own Address Space

thats not what i said -- i think you missunderstood me

what i said is:
a single copy is sufficient for several processes as long as they are loaded into the same virtual address which is what windows does

however, its not uncommon for processes to have many different shared libs loaded at the same time, therefore, it is impossible to gaurentee that the same address will be availible in all process address spaces, so sometimes another copy is needed to place at an alternate address for processes where the prefered address is already in use (either by the application itself, or another shared lib)

what i suggested was like this: every shared lib is loaded into memory at an address (iirc windows dlls have a 'prefered address' field in the header)

next time a process loads that same lib, the OS trys to load it at whatever address it is loaded in the other process -- if it is successful, it will use the same copy (using shared memory), if not, it will create another copy, assigning it to another address

the 3rd time, it checks if either of the existing copies can be used, it will create a third copy only if neither of the existing copies can be used (both addresses are in use -- a much more rare condition)


if the OS allows segments(not allowed in my OS -- or most others), its possible to make it appear at a single logical address by placeing each shared lib in its own segment (in the LDT -- actually the best use i can imagine for the LDT)

or if the OS requires all shared libs to use PIC code (very difficult to implement in PMode) then it wont matter where in virtual memory it is loaded (and a single instance can be used regardless of logical address )

[prajwal]

thats not what i said -- i think you missunderstood me

Ok!... yes i have misunderstood your explanation...

what i suggested was like this: every shared lib is loaded into memory at an address (iirc windows dlls have a 'prefered address' field in the header)

next time a process loads that same lib, the OS trys to load it at whatever address it is loaded in the other process -- if it is successful, it will use the same copy (using shared memory), if not, it will create another copy, assigning it to another address

the 3rd time, it checks if either of the existing copies can be used, it will create a third copy only if neither of the existing copies can be used (both addresses are in use -- a much more rare condition)

This looks very interseting and its tempting me to implement it in that way right now....
But I'll definitely Try this approach once I start considering Performance of my OS...

if the OS allows segments(not allowed in my OS -- or most others), its possible to make it appear at a single logical address by placeing each shared lib in its own segment (in the LDT -- actually the best use i can imagine for the LDT)

Sorry! this part i'm unable to understand clearly.... may be because of my less knowledge in this Field...[/quote]

[JAAman]

if you support useing non-flat segments, you can place shared libraries into separate segments -- that way the segmentation modifies the address space so that they all appear to be at the same address -- regardless of where in the linear address space they actually are, they are at the same logical address:

shared lib expects to be at address 0
lib is loaded at address 0x0123 4567

setup a segment where cs.base = 0x0123 4567
then to enter the shared lib you will need to use a far call ( lib.CS:entry), and the lib will think that it is located at address 0, rather than at address 0x0123 4567

in this way, all processes could use the same copy, because it thinks its running at the same address, because it uses its own segment registers

however, most hobby OSs (including mine) dont allow non-flat segments (and this would not work in LMode)

[prajwal]

That's a great Solution !!... But a tedious implementation for me now.... I'll stick to previous approach....

Thanks for clearly explaning the various concepts...