PE vs ELF for HobbyOS Usermode - In Your Opinion

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
Hoozim
Member
Member
Posts: 53
Joined: Fri Jul 23, 2010 8:26 am

PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by Hoozim »

Hello,

In your opinion, do you prefer PE32 or ELF32? I am speaking in terms of relocations, importing and exporting. My OS currently supports all of those with PE32. I am considering supporting ELF too with those requirements. I want to know if it is worth my time to support both before I delve into technical documentation. The reason I am considering ELF support is to simplify porting programd such as GCC which prefers the ELF target on Linux.

Hoozim
User avatar
Nessphoro
Member
Member
Posts: 308
Joined: Sat Apr 30, 2011 12:50 am

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by Nessphoro »

I prefer ELF because I associate PE with "evil" Microsoft, and who knows what might happen in their lawyer's head.
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by Brendan »

Hi,
Hoozim wrote:In your opinion, do you prefer PE32 or ELF32? I am speaking in terms of relocations, importing and exporting. My OS currently supports all of those with PE32. I am considering supporting ELF too with those requirements. I want to know if it is worth my time to support both before I delve into technical documentation. The reason I am considering ELF support is to simplify porting programd such as GCC which prefers the ELF target on Linux.
In my opinion, if you're writing a Windows clone use PE32, and if you're writing a Unix clone use ELF. If you're not writing a clone make up your own executable format specifically designed for your OS, and then see if PE or ELF is "close enough" to what you actually do want.

Also note that (for some OS designs - e.g. anything that isn't "monolithic with kernel modules") relocations can be avoided, and importing/exporting is unnecessary and leads to dependency hell. Something like "flat binary with custom header" is simpler, more efficient and prevents all the problems.

In theory, shared libraries/DLLs can save RAM; but in practice it doesn't because most shared libraries/DLLs are only used by one process and a compiler can't effectively optimise (e.g. to remove unused code) so it actually tends to use more RAM than it saves and makes things worse (slower and more bloated). In theory shared libraries/DLLs can also make it easier to fix bugs (e.g. just fix the bug in the library and all software that uses the library benefits from the fix immediately); but in practice this often works in reverse (change the library and break all software that relied on the old behaviour). To get around that problem most systems have some sort of version numbering (e.g. so that "application foo" relies on "version 1.2.3 of shared library bar", which just makes the first problem worse (e.g. several processes using different versions of the same shared library) and prevents the theoretical "just fix the bug in the library and all software that uses the library benefits" advantage.

The only shared library that actually makes sense (as it's far more likely to be used by multiple processes) is the compiler's run-time library, but if each process only ever uses one library then you can give it a fix virtual address and use a jump table instead of dynamic linking.


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
User avatar
Combuster
Member
Member
Posts: 9301
Joined: Wed Oct 18, 2006 3:45 am
Libera.chat IRC: [com]buster
Location: On the balcony, where I can actually keep 1½m distance
Contact:

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by Combuster »

PE is both stuffed and non-extendible beyond a lot of windows-specific things, which makes parsing it more complicated and doesn't allow you to do much beyond the original specifications. ELF is mostly OS-agnostic and has sufficient opportunities to attach any custom data to it.

Supporting another file format because the code expects it is basically nothing other than feeding candy to a naughty child.
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]
User avatar
Solar
Member
Member
Posts: 7635
Joined: Thu Nov 16, 2006 12:01 pm
Location: Germany
Contact:

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by Solar »

Brendan wrote:In theory, shared libraries/DLLs can save RAM; but in practice it doesn't because most shared libraries/DLLs are only used by one process and a compiler can't effectively optimise (e.g. to remove unused code) so it actually tends to use more RAM than it saves and makes things worse (slower and more bloated).
A shared library should only be built by a project that has the purpose of providing a shared library. LibC, Boost are good examples for shared libraries. MyCompanyFooLib and MyCompanyBarLib are usually bad examples.
Brendan wrote:In theory shared libraries/DLLs can also make it easier to fix bugs (e.g. just fix the bug in the library and all software that uses the library benefits from the fix immediately); but in practice this often works in reverse (change the library and break all software that relied on the old behaviour). To get around that problem most systems have some sort of version numbering (e.g. so that "application foo" relies on "version 1.2.3 of shared library bar", which just makes the first problem worse (e.g. several processes using different versions of the same shared library) and prevents the theoretical "just fix the bug in the library and all software that uses the library benefits" advantage.
A side effect of the AmigaOS LibraryBase approach (making function access go through a jump table) was that a single library could provide several versions of its API.

Consider (after reading above link) version 1 of some library having the functions:
  • Foo()
  • Bar()
In version 2, you add version Baz(), and fix a bug in Foo(), which makes the function behave in a non-compatible way, yielding Foo() version 2. Your new jump table looks like:
  • Foo() (version 2)
  • Bar()
  • Baz()
Any application calling OpenLibrary( "YourLib", 2 ) would get this new jump table. But to keep backward compatibility, you keep Foo() in the library, and provide another jump table identical to that of library version 1. A call of OpenLibrary( "YourLib", 1 ) would get the old jump table.

Advantage: You have to keep YourLib.library in memory only once, regardless of which versions are requested. The library might be a bit bigger, as it retains old API, but it is still more efficient than having both versions of the library on disk and in memory.

Anyway, that's what I would have copied in my OS; perhaps it inspires one of you. If it doesn't, feel free to ignore my rambling. 8)
Every good solution is obvious once you've found it.
Hoozim
Member
Member
Posts: 53
Joined: Fri Jul 23, 2010 8:26 am

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by Hoozim »

My goal is to modularize my OS so importing and exporting is important. How easy is importing and exporting with ELF?

Hoozim
User avatar
bluemoon
Member
Member
Posts: 1761
Joined: Wed Dec 01, 2010 3:41 am
Location: Hong Kong

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by bluemoon »

On ELF, the symbol tables is as simple as an array.
There are two table, or array, of most interest:
offset = symbol_offset[index];
name = symbol_name[index];

by matching the name and offset you can do import/export relatively straightforward.
Hoozim
Member
Member
Posts: 53
Joined: Fri Jul 23, 2010 8:26 am

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by Hoozim »

So ELF is a lot like PE in terms of dynamic linking which uses three arrays. I think I will go ahead and implement ELF.

Thank you for all of your help everyone!

Hoozim
Rudster816
Member
Member
Posts: 141
Joined: Thu Jun 17, 2010 2:36 am

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by Rudster816 »

If you read the PE spec, you'll realize that a lot of things are Windows specific. The spec (at the least the one I got from Microsofts website) requires a DOS executable stub at the beginning (which is why you'll always find that "This program can't be run in DOS mode" string in PE's). This is just one of the several things that are compltely useless for non Windows\ReactOS operating systems. ELF is basically as Machine\OS independent as it can physically be.

@Solar

While good in theory, keeping seperate versions of DLL's, or just different version of rountines inside the same DLL is good in theory, after awhile it can take up a huge chunk of memory\disk space. My .NET assembly cache is at 2GB right now. Imagine if I had several .NET programs open all running different versions, just the memory for the shared DLL's could literally take up 100's of MB worth of RAM.

I think it would be much better to plan your API's well and stick to well defined behavior. So long as behavior of the functions are well documented\defined, bugfixes and small updates should be possible while not breaking anything that isn't buggy\broken. You can then add new functions as the need arises (rather than updating existing ones) and deprecate old ones as needed. So long as you don't try to maintain compatibility with ancient software (*cough* Windows *cough*) your interface should stay fairly clean.
Hoozim
Member
Member
Posts: 53
Joined: Fri Jul 23, 2010 8:26 am

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by Hoozim »

I agree, the PE spec. does contain a lot of garbage for the hobbyos developer. The only reason I implemented it was because I had previous experience with it from I started OSDev which meant not reading an entire spec, just the little details I had forgotten.

Hoozim
User avatar
bluemoon
Member
Member
Posts: 1761
Joined: Wed Dec 01, 2010 3:41 am
Location: Hong Kong

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by bluemoon »

Hoozim wrote:So ELF is a lot like PE in terms of dynamic linking which uses three arrays.
Even with ELF, for dynamic linking you still need 3 or 4 arrays:
- symbol's offset(SHT_SYNSYM)
- symbol's name(SHT_SYNSYM)
- relocation table(SHT_REL*).
- optionally, to link against multiple objects, your import table, which to patch with external offset if symbol's offset is zero.
User avatar
qw
Member
Member
Posts: 792
Joined: Mon Jan 26, 2009 2:48 am

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by qw »

I was wondering how position-independent code fits into the discussion. I never heard of PIC in PE. Does anyone here use PIC in his/her hobby OS?
User avatar
bluemoon
Member
Member
Posts: 1761
Joined: Wed Dec 01, 2010 3:41 am
Location: Hong Kong

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by bluemoon »

How does PIC has anything to do with executable format?
The compiler generate PIC code, usually with specially tweaked register usages, and the linker put it into the format as-is.
User avatar
qw
Member
Member
Posts: 792
Joined: Mon Jan 26, 2009 2:48 am

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by qw »

bluemoon wrote:How does PIC has anything to do with executable format?
Well, that was my question actually. Maybe PIC is easier to implement in one format than the other.

BTW I found that 64-bits Windows does use position-independent code, so now I have heard of PIC in PE.
User avatar
JamesM
Member
Member
Posts: 2935
Joined: Tue Jul 10, 2007 5:27 am
Location: York, United Kingdom
Contact:

Re: PE vs ELF for HobbyOS Usermode - In Your Opinion

Post by JamesM »

bluemoon wrote:
Hoozim wrote:So ELF is a lot like PE in terms of dynamic linking which uses three arrays.
Even with ELF, for dynamic linking you still need 3 or 4 arrays:
- symbol's offset(SHT_SYNSYM)
- symbol's name(SHT_SYNSYM)
- relocation table(SHT_REL*).
- optionally, to link against multiple objects, your import table, which to patch with external offset if symbol's offset is zero.
Plus the PLT and GOT for lazy linking.
Post Reply