Page 1 of 1

PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Wed Mar 28, 2012 9:18 pm
by Hoozim
Hello,

In your opinion, do you prefer PE32 or ELF32? I am speaking in terms of relocations, importing and exporting. My OS currently supports all of those with PE32. I am considering supporting ELF too with those requirements. I want to know if it is worth my time to support both before I delve into technical documentation. The reason I am considering ELF support is to simplify porting programd such as GCC which prefers the ELF target on Linux.

Hoozim

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Wed Mar 28, 2012 9:55 pm
by Nessphoro
I prefer ELF because I associate PE with "evil" Microsoft, and who knows what might happen in their lawyer's head.

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Wed Mar 28, 2012 10:58 pm
by Brendan
Hi,
Hoozim wrote:In your opinion, do you prefer PE32 or ELF32? I am speaking in terms of relocations, importing and exporting. My OS currently supports all of those with PE32. I am considering supporting ELF too with those requirements. I want to know if it is worth my time to support both before I delve into technical documentation. The reason I am considering ELF support is to simplify porting programd such as GCC which prefers the ELF target on Linux.
In my opinion, if you're writing a Windows clone use PE32, and if you're writing a Unix clone use ELF. If you're not writing a clone make up your own executable format specifically designed for your OS, and then see if PE or ELF is "close enough" to what you actually do want.

Also note that (for some OS designs - e.g. anything that isn't "monolithic with kernel modules") relocations can be avoided, and importing/exporting is unnecessary and leads to dependency hell. Something like "flat binary with custom header" is simpler, more efficient and prevents all the problems.

In theory, shared libraries/DLLs can save RAM; but in practice it doesn't because most shared libraries/DLLs are only used by one process and a compiler can't effectively optimise (e.g. to remove unused code) so it actually tends to use more RAM than it saves and makes things worse (slower and more bloated). In theory shared libraries/DLLs can also make it easier to fix bugs (e.g. just fix the bug in the library and all software that uses the library benefits from the fix immediately); but in practice this often works in reverse (change the library and break all software that relied on the old behaviour). To get around that problem most systems have some sort of version numbering (e.g. so that "application foo" relies on "version 1.2.3 of shared library bar", which just makes the first problem worse (e.g. several processes using different versions of the same shared library) and prevents the theoretical "just fix the bug in the library and all software that uses the library benefits" advantage.

The only shared library that actually makes sense (as it's far more likely to be used by multiple processes) is the compiler's run-time library, but if each process only ever uses one library then you can give it a fix virtual address and use a jump table instead of dynamic linking.


Cheers,

Brendan

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Thu Mar 29, 2012 1:34 am
by Combuster
PE is both stuffed and non-extendible beyond a lot of windows-specific things, which makes parsing it more complicated and doesn't allow you to do much beyond the original specifications. ELF is mostly OS-agnostic and has sufficient opportunities to attach any custom data to it.

Supporting another file format because the code expects it is basically nothing other than feeding candy to a naughty child.

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Thu Mar 29, 2012 1:55 am
by Solar
Brendan wrote:In theory, shared libraries/DLLs can save RAM; but in practice it doesn't because most shared libraries/DLLs are only used by one process and a compiler can't effectively optimise (e.g. to remove unused code) so it actually tends to use more RAM than it saves and makes things worse (slower and more bloated).
A shared library should only be built by a project that has the purpose of providing a shared library. LibC, Boost are good examples for shared libraries. MyCompanyFooLib and MyCompanyBarLib are usually bad examples.
Brendan wrote:In theory shared libraries/DLLs can also make it easier to fix bugs (e.g. just fix the bug in the library and all software that uses the library benefits from the fix immediately); but in practice this often works in reverse (change the library and break all software that relied on the old behaviour). To get around that problem most systems have some sort of version numbering (e.g. so that "application foo" relies on "version 1.2.3 of shared library bar", which just makes the first problem worse (e.g. several processes using different versions of the same shared library) and prevents the theoretical "just fix the bug in the library and all software that uses the library benefits" advantage.
A side effect of the AmigaOS LibraryBase approach (making function access go through a jump table) was that a single library could provide several versions of its API.

Consider (after reading above link) version 1 of some library having the functions:
  • Foo()
  • Bar()
In version 2, you add version Baz(), and fix a bug in Foo(), which makes the function behave in a non-compatible way, yielding Foo() version 2. Your new jump table looks like:
  • Foo() (version 2)
  • Bar()
  • Baz()
Any application calling OpenLibrary( "YourLib", 2 ) would get this new jump table. But to keep backward compatibility, you keep Foo() in the library, and provide another jump table identical to that of library version 1. A call of OpenLibrary( "YourLib", 1 ) would get the old jump table.

Advantage: You have to keep YourLib.library in memory only once, regardless of which versions are requested. The library might be a bit bigger, as it retains old API, but it is still more efficient than having both versions of the library on disk and in memory.

Anyway, that's what I would have copied in my OS; perhaps it inspires one of you. If it doesn't, feel free to ignore my rambling. 8)

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Thu Mar 29, 2012 3:15 pm
by Hoozim
My goal is to modularize my OS so importing and exporting is important. How easy is importing and exporting with ELF?

Hoozim

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Thu Mar 29, 2012 3:23 pm
by bluemoon
On ELF, the symbol tables is as simple as an array.
There are two table, or array, of most interest:
offset = symbol_offset[index];
name = symbol_name[index];

by matching the name and offset you can do import/export relatively straightforward.

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Thu Mar 29, 2012 3:50 pm
by Hoozim
So ELF is a lot like PE in terms of dynamic linking which uses three arrays. I think I will go ahead and implement ELF.

Thank you for all of your help everyone!

Hoozim

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Thu Mar 29, 2012 4:16 pm
by Rudster816
If you read the PE spec, you'll realize that a lot of things are Windows specific. The spec (at the least the one I got from Microsofts website) requires a DOS executable stub at the beginning (which is why you'll always find that "This program can't be run in DOS mode" string in PE's). This is just one of the several things that are compltely useless for non Windows\ReactOS operating systems. ELF is basically as Machine\OS independent as it can physically be.

@Solar

While good in theory, keeping seperate versions of DLL's, or just different version of rountines inside the same DLL is good in theory, after awhile it can take up a huge chunk of memory\disk space. My .NET assembly cache is at 2GB right now. Imagine if I had several .NET programs open all running different versions, just the memory for the shared DLL's could literally take up 100's of MB worth of RAM.

I think it would be much better to plan your API's well and stick to well defined behavior. So long as behavior of the functions are well documented\defined, bugfixes and small updates should be possible while not breaking anything that isn't buggy\broken. You can then add new functions as the need arises (rather than updating existing ones) and deprecate old ones as needed. So long as you don't try to maintain compatibility with ancient software (*cough* Windows *cough*) your interface should stay fairly clean.

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Thu Mar 29, 2012 4:23 pm
by Hoozim
I agree, the PE spec. does contain a lot of garbage for the hobbyos developer. The only reason I implemented it was because I had previous experience with it from I started OSDev which meant not reading an entire spec, just the little details I had forgotten.

Hoozim

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Fri Mar 30, 2012 1:00 am
by bluemoon
Hoozim wrote:So ELF is a lot like PE in terms of dynamic linking which uses three arrays.
Even with ELF, for dynamic linking you still need 3 or 4 arrays:
- symbol's offset(SHT_SYNSYM)
- symbol's name(SHT_SYNSYM)
- relocation table(SHT_REL*).
- optionally, to link against multiple objects, your import table, which to patch with external offset if symbol's offset is zero.

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Fri Mar 30, 2012 1:32 am
by qw
I was wondering how position-independent code fits into the discussion. I never heard of PIC in PE. Does anyone here use PIC in his/her hobby OS?

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Fri Mar 30, 2012 1:53 am
by bluemoon
How does PIC has anything to do with executable format?
The compiler generate PIC code, usually with specially tweaked register usages, and the linker put it into the format as-is.

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Fri Mar 30, 2012 2:15 am
by qw
bluemoon wrote:How does PIC has anything to do with executable format?
Well, that was my question actually. Maybe PIC is easier to implement in one format than the other.

BTW I found that 64-bits Windows does use position-independent code, so now I have heard of PIC in PE.

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Posted: Fri Mar 30, 2012 5:01 am
by JamesM
bluemoon wrote:
Hoozim wrote:So ELF is a lot like PE in terms of dynamic linking which uses three arrays.
Even with ELF, for dynamic linking you still need 3 or 4 arrays:
- symbol's offset(SHT_SYNSYM)
- symbol's name(SHT_SYNSYM)
- relocation table(SHT_REL*).
- optionally, to link against multiple objects, your import table, which to patch with external offset if symbol's offset is zero.
Plus the PLT and GOT for lazy linking.