Page 2 of 3

Re: PE vs ELF

Posted: Sun Aug 10, 2008 11:50 am
by AJ
ru2aqare wrote:On the other hand, I still want to develop an ELF loader for my OS (why not?), but it has some concepts unfamiliar to the average Windows developer.
...
Bottom line: if you want something simple and fast, develop a custom image format for your OS.
To me this seems very much like "I don't understand the documentation, therefore I'll make up my own format." The nice thing about PE and ELF are that they are very versatile and have mature toolchains associated with them. The documentation for ELF is also very full - I don't know about PE.

I would only recommend using your own format if you want to do something that an existing format can't handle. I also wouldn't fancy having to recompile all my binaries each time another feature needed adding to my new format.

Cheers,
Adam

Re: PE vs ELF

Posted: Sun Aug 10, 2008 12:48 pm
by ru2aqare
AJ wrote:To me this seems very much like "I don't understand the documentation, therefore I'll make up my own format."
While it may be true that I don't understand the ELF documentation completely, I am not using (and never wanted to use) a custom format. I use the PE format because it has an existing toolchain, and it has all the features I currently need. It just seems to me that ELF is more complex.

Hope this time I expressed myself more clearly.

Re: PE vs ELF

Posted: Sun Aug 10, 2008 4:09 pm
by bewing
The only pieces of the toolchain that need to be modified are the linker and the debugger. They are the only ones that handle the raw image format. Everything else can still use standard object files.
And all you have to have in your image format is everything in PE + everything in ELF, and then you don't have to worry about "oh it will be so painful if I have to add another feature to my image format".

I think the important point is that neither ELF nor PE is as slick and pretty and understandable as it really should be.

Re: PE vs ELF

Posted: Mon Aug 11, 2008 9:40 am
by z180
I know that the PE format was inspired by System V R3 COFF because ELF is from System V R4 from 1990 at this time NT was already running at MS.
With the mach-o format the next/apple/mach os do interesting things wich dont fit into my head now(need lunch break).
In my os I used elf and a variation of the RDF format that was documented in the NASM assembler

Re: PE vs ELF

Posted: Mon Aug 25, 2008 12:06 pm
by Jeko
Have you any informations about Mach-O file format? I would like to read something about it, maybe it can be a good format for my OS.

However between ELF and PE, I think ELF is better for my OS, because I want PIC code to share libraries in memory. And if I share a library in memory with PE, many times I must relocate it and so it's really slow.

Re: PE vs ELF

Posted: Mon Aug 25, 2008 9:06 pm
by Brynet-Inc
Jeko wrote:Have you any informations about Mach-O file format? I would like to read something about it, maybe it can be a good format for my OS.
It's common courtesy to first go out and search for things yourself... nobody here wants to spoon feed beginners, you need to be at least somewhat self-sufficient.

http://en.wikipedia.org/wiki/Mach-O
http://developer.apple.com/documentatio ... h-O.5.html
http://developer.apple.com/documentatio ... rence.html

Re: PE vs ELF

Posted: Thu Sep 18, 2008 8:12 pm
by Love4Boobies
PE is highly superior to ELF, yet ELF is very well documented. I'd go for PE.

P.S: The Windows arguments are plain stupid :D

Re: PE vs ELF

Posted: Fri Sep 19, 2008 8:41 am
by Jeko
Love4Boobies wrote:PE is highly superior to ELF
Why? You're saying something without any argumentation...
ELF is an extension of the PE format (COFF), so PE can't be highly superior to ELF. You can think it's superior, but not highly superior...

However, can you say us the reasons of your opinion?

Re: PE vs ELF

Posted: Sun Sep 21, 2008 2:50 am
by elderK
Jeko, I've told you before - ELF is the way to go, for various reasons.

But, I find myself in the mood to rant, so I do - and thus, this post.

As for those who say PE is "more efficient" or "better on RAM", no, not really. It has some minor advantages, such as being able to resolve symbols via ordinal numbers - instead of by name. However, afaik, Microsoft now looks DOWN on people using ordinals exclusively - since those numbers are difficult to keep uniform across Shared library releases (or so I have read...).

Now, onto efficiency - in relation to memory. PE is less efficient in the long term. Why? Because as soon as a DLL has to be rebased, it's text is no longer sharable amongst other programs. .DLLs all have a preferred base address and, are gnerally only relocated when that address is not available. Sure - it's possible that the relocated DLL could be shared to another program, in the case that the rebased address is available but, that can be argued to be a 'special case'.

Compare this to ELF - where code can be truly position-independent _and_ sharable. Data may need to be copied or otherwise made exclusive - but the code itself, can be shared EVEN when it HAS been relocated. How? Why?

Two useful tables : GOT, PLT.

As PIC calls are resolved via the PLT - and variables resolved via the GOT, the code does not NEED necessarily ever have to be modified - aslong as the offsets from the code AND the GOT/PLT are consistent. So says the ELF documentation. Also, when code IS relocated - it isn't the code that is modified - it is the GOT/PLT. IIRC, both of which are private to whatever ELF is in question.

Constrast that:
PE, relocated DLL to a new base - Code has been copied, modified - cannot be shared.
VS
ELF, relocated DSO, plt/got has been changed - Code can still be shared, etc.

I'll set my watch and warrant on the GOT/PLT pair being smaller than the code itself. :P
So, in this case - ELF wins.

Also, when it comes to the specific designs - ELF is much, much more general. PE/COFF is mostly specific
to Windows (apart from the few projects that make use of it). There is a LOT of crap that your OS is unlikely to need.
Ie: Major/Minor Subsystem Identifiers, Major/Minor bla bla bla. Not to mention the MSDOS stub, etc.

While ELF binaries maybe larger than the equivalent PE, at least the space is spent wisely (storing relocation entries, symbol tables, a lot of which can be stripped...) vs. pointless MSDOS stubs and 'redundant' COFF headers.

ELF PIC Executables are simple.
ELF Shared Objects are simple.
Dynamic Linkers for both, are actually quite simple.
ELF Relocatables (.o) are more of a pain in the @$$ but, once you have a DL working, is quite simple.

Not to mention, tools to produce ELF are available on virtually every platform imaginable without major armtwisting.
Except of course, under Windows and even then, you have Cygwin to ease your pain.

~elderK
EDIT: ELF is _NOT_ an extension of the PE/COFF format. PE is an extension of the COFF format, ELF was new.
XCOFF, ECOFF, PE/C0OFF, COFF - those are COFF based. ELF, isn't.

Re: PE vs ELF

Posted: Sun Sep 21, 2008 7:18 am
by ru2aqare
elderK wrote:...
Now, onto efficiency - in relation to memory. PE is less efficient in the long term. Why? Because as soon as a DLL has to be rebased, it's text is no longer sharable amongst other programs. .DLLs all have a preferred base address and, are gnerally only relocated when that address is not available. Sure - it's possible that the relocated DLL could be shared to another program, in the case that the rebased address is available but, that can be argued to be a 'special case'.
That is why we have the rebase tool. You can calculate (either manually or automatically) the base addresses for each shared library (DLL) so that relocation will almost never occur. And if it does not need to be performed, there is no reason why the code can't be shared among processes. It is even less an issue on x64, since it has RIP-relative addressing, eliminating most of the relocations.
elderK wrote: Compare this to ELF - where code can be truly position-independent _and_ sharable. Data may need to be copied or otherwise made exclusive - but the code itself, can be shared EVEN when it HAS been relocated. How? Why?

Two useful tables : GOT, PLT.
And in the process you lose one register, which must be reserved to keep track of these tables. It is less of an issue on x64, since it has plenty of registers anyway.
elderK wrote: Also, when it comes to the specific designs - ELF is much, much more general. PE/COFF is mostly specific
to Windows (apart from the few projects that make use of it). There is a LOT of crap that your OS is unlikely to need.
Ie: Major/Minor Subsystem Identifiers, Major/Minor bla bla bla. Not to mention the MSDOS stub, etc.

While ELF binaries maybe larger than the equivalent PE, at least the space is spent wisely (storing relocation entries, symbol tables, a lot of which can be stripped...) vs. pointless MSDOS stubs and 'redundant' COFF headers.
You can safely ignore all of these, if you don't use them. And they occupy like 512 bytes at most - not a big waste of memory to cry over. PE files usually don't have a symbol table, however (symbol table != exported symbols). I don't know whether it is good or not - you only need the symbols for a stack trace anyway.
elderK wrote: ELF PIC Executables are simple.
ELF Shared Objects are simple.
PE files are so simple that I have a loader in like 50 lines, which relocates sections with respect to paging.

Edit:
elderK wrote: Except of course, under Windows and even then, you have Cygwin to ease your pain.
The Cygwin toolchain by default produces PE files. To get ELF output, you need to recompile at least binutils. Probably not worth the effort for a smaller project, when you can use the Windows native file format.

Re: PE vs ELF

Posted: Mon Sep 22, 2008 2:35 am
by Combuster
PE files are so simple that I have a loader in like 50 lines, which relocates sections with respect to paging.
My ELF loader is about 150 lines of assembly. Which means that a C version would be well under the 50 lines.

Re: PE vs ELF

Posted: Mon Sep 22, 2008 3:30 pm
by Jeko
ru2aqare wrote:PE files are so simple that I have a loader in like 50 lines, which relocates sections with respect to paging.
In 50 lines do you handle also dynamic linking? Can you share your source code?

Combuster, the question is the same for you.

Re: PE vs ELF

Posted: Mon Sep 22, 2008 3:58 pm
by ru2aqare
Jeko wrote:
ru2aqare wrote:PE files are so simple that I have a loader in like 50 lines, which relocates sections with respect to paging.
In 50 lines do you handle also dynamic linking? Can you share your source code?

Combuster, the question is the same for you.
Around fifty lines (...not including headers), but without dynamic linking. I don't use dynamic linking in my boot loader, however I should have a working dynamic linker lying around somewhere... *goes searching the SVN repository*

For the time being, here is the loader (I hope nothing is missing from the rar).

Re: PE vs ELF

Posted: Tue Sep 23, 2008 12:40 am
by Combuster
In 50 lines do you handle also dynamic linking?
No I don't do dynamic linking yet. The point is mainly that it gets a job done.
Can you share your source code?
It's already shared, for the past two years even: [ My OS ]

Re: PE vs ELF

Posted: Wed Sep 24, 2008 3:44 am
by elderK
As for ELFs, no, you dont necessarily have to lose out on a single register.
It depends if it is full PIC code, or simply relocatable code and target architecture.

512Bytes isnt a big thing to get upset about?
MY I REMIND YOU THAT 512B can BOOT the freaking machine?
That 512B can store SHITLOADS of data?
Nodes, all kinds of stuff?

Yeah, its so totally worth not worrying about - Ill just waste MORE memory in some way.
Phft. Shame on you.

As for the REBASE tool, good for you.
THe point is that you must precalculate the base - before use. Otherwise, the OS has to modify it, which makes the text unable to be shared, read the MS Spec and see for yourself.

Where as ELF can go ANYWHERE, without precalculation.
PE without rebasing, is faster for initial load.
PE with rebasing at loadtime - slower.

But hey, believe want you want.

FINALLY though, when it comes to loading static ELFs / PE, joy :P
Read the headers, load the sections. :P Done.

~K