Page 1 of 2
flat binary
Posted: Wed Apr 04, 2012 2:12 am
by stanko51
Hi, I'm not sure about this, so I'm just asking.
Let's say I have a flat binary, compiled and linked with -oformat=binary. i don't specify anything else to the linker.
Can it be copied anywhere in memory (from HDD for example), and just be executed, either by a asm call or by task switching (providing eip with the address) ?
When tested it works fine, but is there some case the code won't work ?
thank you
Re: flat binary
Posted: Wed Apr 04, 2012 2:40 am
by AndreaOrru
You should use -fPIC if you want to be able to execute it anywhere you like.
Re: flat binary
Posted: Wed Apr 04, 2012 2:42 am
by assembler01
stanko51 wrote:Can it be copied anywhere in memory (from HDD for example), and just be executed, either by a asm call or by task switching (providing eip with the address) ?
Yes.
stanko51 wrote:When tested it works fine, but is there some case the code won't work ?
No.
Re: flat binary
Posted: Wed Apr 04, 2012 3:18 am
by Brendan
Hi,
stanko51 wrote:Let's say I have a flat binary, compiled and linked with -oformat=binary. i don't specify anything else to the linker.
Can it be copied anywhere in memory (from HDD for example), and just be executed, either by a asm call or by task switching (providing eip with the address) ?
If you use paging and/or segmentation to make sure the code runs from the virtual address it's linked to expect, then there's no problem regardless of what physical address you loaded the code at. If you don't do either of these things and run the code from a virtual address that it isn't linked to expect; then any instruction that relies on a fixed address will cause problems. This includes simple instructions like "mov esi,helloWorldString" and more complex situations like "call [functionTable + eax*4]".
The "-fPIC" thing tells the compiler not to generate instructions that rely on a fixed address. This could mean forcing the use of "RIP relative" addressing in 64-bit 80x86 code (e.g. "lea rsi,[rip+helloWorldString-$]" instead of "mov rsi,helloWorldString"), or using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
Cheers,
Brendan
Re: flat binary
Posted: Wed Apr 04, 2012 4:58 am
by Solar
Brendan wrote:...using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
I wonder...
At the age of micro-ops, register renaming and other such stunts, how big
is the performance penalty of PIC code on modern x86 / x64 cores?
Re: flat binary
Posted: Wed Apr 04, 2012 8:56 am
by Nable
> mov esi,[rbx+helloWorldString]
My message may be rude but maybe you wanted to say
lea esi, [ebx+helloWorldString]
?
> At the age of micro-ops, register renaming and other such stunts, how big is the performance penalty of PIC code on modern x86 / x64 cores?
x64 has RIP-relative addressing, so, looks like this overhead is very small.
For x86 overhead (IMHO) is significant: although we have shadow registers (register renaming), i don't think that CPUs can easily translate the code to use more registers. Execution planning and register allocation is a difficult task even for compilers, and you expect CPU to transform code on the fly, i don't believe in it.
Overuse of PIC is one of the things that i hate most in *nix systems, windows style of relocations is (IMHO) better than using -fPIC for all dynamic libraries. Although, if your application is often stuck waiting for I/O with slower devices (and that's a common situation for GUI apps, file or web servers, etc) then you'll just never notice this penalty of PIC.
Re: flat binary
Posted: Wed Apr 04, 2012 9:30 am
by turdus
Brendan wrote:...or using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
Not necessairly. In protmode you can point code and data segment to the address of the code, and use only es,fs,gs for flat memory access. This implies address must be at paragraph border though, but saving a register worth it I think.
Re: flat binary
Posted: Wed Apr 04, 2012 10:47 am
by bubach
turdus wrote:Brendan wrote:...or using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
Not necessairly. In protmode you can point code and data segment to the address of the code, and use only es,fs,gs for flat memory access. This implies address must be at paragraph border though, but saving a register worth it I think.
yeah, thats why he said this just a few paragraphs above:
Brendan wrote:If you use paging and/or segmentation to make sure the code runs from the virtual address it's linked to expect, then there's no problem regardless of what physical address you loaded the code at.
Re: flat binary
Posted: Wed Apr 04, 2012 11:34 am
by Combuster
Solar wrote:At the age of micro-ops, register renaming and other such stunts, how big is the performance penalty of PIC code on modern x86 / x64 cores?
On x86 it is obviously a problem: PIC implies that you lose one out of the six remaining registers for computational tasks. That means in turn that you need
memory accesses to swap that storage out, meaning more and longer instructions (with corresponding cache effects and decoder pressure), and more uop pressure on the loadstore and address generation units for the stores to happen. Those same units are also responsible for doing EBX-relative addressing. If that load is added onto code that demands of the same processor components, the resulting performance loss is very noticeable (i.e. more than the percentage of registers lost). The first benchmark I found mentions 6% performance loss in practice. (compared to 0.02% on amd64)
On amd64 the entire register pressure effect is simply gone since you can use RIP addressing.
Re: flat binary
Posted: Wed Apr 04, 2012 12:50 pm
by Solar
@ Combuster:
Thanks for the benchmark data. (The
theory was clear to me, I just wondered what the real-world impact would be like.)
Nable wrote:Overuse of PIC is one of the things that i hate most in *nix systems, windows style of relocations is (IMHO) better than using -fPIC for all dynamic libraries.
PIC is clearly the superior technology compared to relocations. The thing that sucks isn't UNIX using PIC, it's x86 not supporting it properly. (On a MC680x0, for example, you won't have any performance penalty, and I'd expect the same from equally modern architectures like PPC or ARM. Actually, anything
but x86.

Re: flat binary
Posted: Wed Apr 04, 2012 1:20 pm
by Nable
> you won't have any performance penalty
won't have any performance penalty compared to what?
"no performance penalty" != "penalty is always included because we don't have a way to live without it".
Just a simple example: loading a constant offset of variable to register. Do you feel the difference: loading constant vs calculating address?
Oh, you can say that global variables are evil, then what about offsets of functions? What about switch-cases2jump_table optimization?
Relocations are just applied after loading and then .reloc can be discarded. Ability to map one instance of library to different addresses (in different processes) is not smth that is used very often. I mean that in Win32 we have DLLs loaded at same addresses in all threads and in rare case when dll cannot be mapped at the same address in new process because this process image is linked to a perverted base, then we can relocate either new process image or create new instance of dll, relocated to address suitable for this process.
---
Oh, it seems that we are turning this thread into flood+offtop+holy-war so i'll try to keep silence, i.e. stop filling this thread with info that topicstarter didn't ask for.
Re: flat binary
Posted: Wed Apr 04, 2012 1:30 pm
by bluemoon
Nable wrote:> you won't have any performance penalty
won't have any performance penalty compared to what?
"no performance penalty" != "penalty is always included because we don't have a way to live without it".
Penalty of using PIC compare to
NOT using PIC.
Nable wrote:I mean that in Win32 we have DLLs loaded at same addresses in all threads and in rare case when dll cannot be mapped at the same address in new process because this process image is linked to a perverted base, then we can relocate either new process image or create new instance of dll, relocated to address suitable for this process.
Think about
Address Space Layout Randomization. On each different relocation you end up with multiple non-shared pages with different patched offset. That can increase stress on memory and cache.
Nable wrote:Oh, it seems that we are turning this thread into flood+offtop+holy-war
No, PIC are critical for running flat binary, and the concern using it (or not) is relavent.
Re: flat binary
Posted: Wed Apr 04, 2012 9:46 pm
by stanko51
Ok, so I'm left with the choice between:
using -fPIC when linking, or
using relocatable executable and relocate it at runtime
is this correct ?
Re: flat binary
Posted: Thu Apr 05, 2012 12:28 am
by Solar
Nable wrote:> you won't have any performance penalty
won't have any performance penalty compared to what?
As in, Intel didn't think about implementing an EIP-relative addressing mode like most (all?) other CPU designers did ("PC-relative in case of the MC68k family), which makes PIC code trivial.
That's why Unix makes such use of PIC: Because the problem with it is specific to the (lack of) x86 addressing modes.
Re: flat binary
Posted: Thu Apr 05, 2012 2:56 am
by Combuster
As in, Intel didn't think about implementing an EIP-relative addressing mode like most (all?) other CPU designers did
Because that issue hardly existed in real mode at all - everyone was using segmentation. People made it a problem by refusing to use it in protected mode as well.