flat binary
flat binary
Hi, I'm not sure about this, so I'm just asking.
Let's say I have a flat binary, compiled and linked with -oformat=binary. i don't specify anything else to the linker.
Can it be copied anywhere in memory (from HDD for example), and just be executed, either by a asm call or by task switching (providing eip with the address) ?
When tested it works fine, but is there some case the code won't work ?
thank you
Let's say I have a flat binary, compiled and linked with -oformat=binary. i don't specify anything else to the linker.
Can it be copied anywhere in memory (from HDD for example), and just be executed, either by a asm call or by task switching (providing eip with the address) ?
When tested it works fine, but is there some case the code won't work ?
thank you
- AndreaOrru
- Member

- Posts: 50
- Joined: Fri Apr 25, 2008 2:50 pm
- Location: New York
Re: flat binary
You should use -fPIC if you want to be able to execute it anywhere you like.
Last edited by AndreaOrru on Wed Apr 04, 2012 2:43 am, edited 1 time in total.
Close the world, txEn eht nepO
-
assembler01
- Member

- Posts: 25
- Joined: Mon Feb 27, 2012 9:46 am
Re: flat binary
Yes.stanko51 wrote:Can it be copied anywhere in memory (from HDD for example), and just be executed, either by a asm call or by task switching (providing eip with the address) ?
No.stanko51 wrote:When tested it works fine, but is there some case the code won't work ?
Talk is cheap, show me the code. - Linus Torvalds
Re: flat binary
Hi,
The "-fPIC" thing tells the compiler not to generate instructions that rely on a fixed address. This could mean forcing the use of "RIP relative" addressing in 64-bit 80x86 code (e.g. "lea rsi,[rip+helloWorldString-$]" instead of "mov rsi,helloWorldString"), or using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
Cheers,
Brendan
If you use paging and/or segmentation to make sure the code runs from the virtual address it's linked to expect, then there's no problem regardless of what physical address you loaded the code at. If you don't do either of these things and run the code from a virtual address that it isn't linked to expect; then any instruction that relies on a fixed address will cause problems. This includes simple instructions like "mov esi,helloWorldString" and more complex situations like "call [functionTable + eax*4]".stanko51 wrote:Let's say I have a flat binary, compiled and linked with -oformat=binary. i don't specify anything else to the linker.
Can it be copied anywhere in memory (from HDD for example), and just be executed, either by a asm call or by task switching (providing eip with the address) ?
The "-fPIC" thing tells the compiler not to generate instructions that rely on a fixed address. This could mean forcing the use of "RIP relative" addressing in 64-bit 80x86 code (e.g. "lea rsi,[rip+helloWorldString-$]" instead of "mov rsi,helloWorldString"), or using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
Cheers,
Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Re: flat binary
I wonder...Brendan wrote:...using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
At the age of micro-ops, register renaming and other such stunts, how big is the performance penalty of PIC code on modern x86 / x64 cores?
Every good solution is obvious once you've found it.
Re: flat binary
> mov esi,[rbx+helloWorldString]
My message may be rude but maybe you wanted to say
lea esi, [ebx+helloWorldString]
?
> At the age of micro-ops, register renaming and other such stunts, how big is the performance penalty of PIC code on modern x86 / x64 cores?
x64 has RIP-relative addressing, so, looks like this overhead is very small.
For x86 overhead (IMHO) is significant: although we have shadow registers (register renaming), i don't think that CPUs can easily translate the code to use more registers. Execution planning and register allocation is a difficult task even for compilers, and you expect CPU to transform code on the fly, i don't believe in it.
Overuse of PIC is one of the things that i hate most in *nix systems, windows style of relocations is (IMHO) better than using -fPIC for all dynamic libraries. Although, if your application is often stuck waiting for I/O with slower devices (and that's a common situation for GUI apps, file or web servers, etc) then you'll just never notice this penalty of PIC.
My message may be rude but maybe you wanted to say
lea esi, [ebx+helloWorldString]
?
> At the age of micro-ops, register renaming and other such stunts, how big is the performance penalty of PIC code on modern x86 / x64 cores?
x64 has RIP-relative addressing, so, looks like this overhead is very small.
For x86 overhead (IMHO) is significant: although we have shadow registers (register renaming), i don't think that CPUs can easily translate the code to use more registers. Execution planning and register allocation is a difficult task even for compilers, and you expect CPU to transform code on the fly, i don't believe in it.
Overuse of PIC is one of the things that i hate most in *nix systems, windows style of relocations is (IMHO) better than using -fPIC for all dynamic libraries. Although, if your application is often stuck waiting for I/O with slower devices (and that's a common situation for GUI apps, file or web servers, etc) then you'll just never notice this penalty of PIC.
Re: flat binary
Not necessairly. In protmode you can point code and data segment to the address of the code, and use only es,fs,gs for flat memory access. This implies address must be at paragraph border though, but saving a register worth it I think.Brendan wrote:...or using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
Re: flat binary
yeah, thats why he said this just a few paragraphs above:turdus wrote:Not necessairly. In protmode you can point code and data segment to the address of the code, and use only es,fs,gs for flat memory access. This implies address must be at paragraph border though, but saving a register worth it I think.Brendan wrote:...or using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
Brendan wrote:If you use paging and/or segmentation to make sure the code runs from the virtual address it's linked to expect, then there's no problem regardless of what physical address you loaded the code at.
- Combuster
- Member

- Posts: 9301
- Joined: Wed Oct 18, 2006 3:45 am
- Libera.chat IRC: [com]buster
- Location: On the balcony, where I can actually keep 1½m distance
- Contact:
Re: flat binary
On x86 it is obviously a problem: PIC implies that you lose one out of the six remaining registers for computational tasks. That means in turn that you need memory accesses to swap that storage out, meaning more and longer instructions (with corresponding cache effects and decoder pressure), and more uop pressure on the loadstore and address generation units for the stores to happen. Those same units are also responsible for doing EBX-relative addressing. If that load is added onto code that demands of the same processor components, the resulting performance loss is very noticeable (i.e. more than the percentage of registers lost). The first benchmark I found mentions 6% performance loss in practice. (compared to 0.02% on amd64)Solar wrote:At the age of micro-ops, register renaming and other such stunts, how big is the performance penalty of PIC code on modern x86 / x64 cores?
On amd64 the entire register pressure effect is simply gone since you can use RIP addressing.
Re: flat binary
@ Combuster:
Thanks for the benchmark data. (The theory was clear to me, I just wondered what the real-world impact would be like.)

Thanks for the benchmark data. (The theory was clear to me, I just wondered what the real-world impact would be like.)
PIC is clearly the superior technology compared to relocations. The thing that sucks isn't UNIX using PIC, it's x86 not supporting it properly. (On a MC680x0, for example, you won't have any performance penalty, and I'd expect the same from equally modern architectures like PPC or ARM. Actually, anything but x86.Nable wrote:Overuse of PIC is one of the things that i hate most in *nix systems, windows style of relocations is (IMHO) better than using -fPIC for all dynamic libraries.
Every good solution is obvious once you've found it.
Re: flat binary
> you won't have any performance penalty
won't have any performance penalty compared to what?
"no performance penalty" != "penalty is always included because we don't have a way to live without it".
Just a simple example: loading a constant offset of variable to register. Do you feel the difference: loading constant vs calculating address?
Oh, you can say that global variables are evil, then what about offsets of functions? What about switch-cases2jump_table optimization?
Relocations are just applied after loading and then .reloc can be discarded. Ability to map one instance of library to different addresses (in different processes) is not smth that is used very often. I mean that in Win32 we have DLLs loaded at same addresses in all threads and in rare case when dll cannot be mapped at the same address in new process because this process image is linked to a perverted base, then we can relocate either new process image or create new instance of dll, relocated to address suitable for this process.
---
Oh, it seems that we are turning this thread into flood+offtop+holy-war so i'll try to keep silence, i.e. stop filling this thread with info that topicstarter didn't ask for.
won't have any performance penalty compared to what?
"no performance penalty" != "penalty is always included because we don't have a way to live without it".
Just a simple example: loading a constant offset of variable to register. Do you feel the difference: loading constant vs calculating address?
Oh, you can say that global variables are evil, then what about offsets of functions? What about switch-cases2jump_table optimization?
Relocations are just applied after loading and then .reloc can be discarded. Ability to map one instance of library to different addresses (in different processes) is not smth that is used very often. I mean that in Win32 we have DLLs loaded at same addresses in all threads and in rare case when dll cannot be mapped at the same address in new process because this process image is linked to a perverted base, then we can relocate either new process image or create new instance of dll, relocated to address suitable for this process.
---
Oh, it seems that we are turning this thread into flood+offtop+holy-war so i'll try to keep silence, i.e. stop filling this thread with info that topicstarter didn't ask for.
Re: flat binary
Penalty of using PIC compare to NOT using PIC.Nable wrote:> you won't have any performance penalty
won't have any performance penalty compared to what?
"no performance penalty" != "penalty is always included because we don't have a way to live without it".
Think about Address Space Layout Randomization. On each different relocation you end up with multiple non-shared pages with different patched offset. That can increase stress on memory and cache.Nable wrote:I mean that in Win32 we have DLLs loaded at same addresses in all threads and in rare case when dll cannot be mapped at the same address in new process because this process image is linked to a perverted base, then we can relocate either new process image or create new instance of dll, relocated to address suitable for this process.
No, PIC are critical for running flat binary, and the concern using it (or not) is relavent.Nable wrote:Oh, it seems that we are turning this thread into flood+offtop+holy-war
Re: flat binary
Ok, so I'm left with the choice between:
using -fPIC when linking, or
using relocatable executable and relocate it at runtime
is this correct ?
using -fPIC when linking, or
using relocatable executable and relocate it at runtime
is this correct ?
Re: flat binary
As in, Intel didn't think about implementing an EIP-relative addressing mode like most (all?) other CPU designers did ("PC-relative in case of the MC68k family), which makes PIC code trivial. That's why Unix makes such use of PIC: Because the problem with it is specific to the (lack of) x86 addressing modes.Nable wrote:> you won't have any performance penalty
won't have any performance penalty compared to what?
Every good solution is obvious once you've found it.
- Combuster
- Member

- Posts: 9301
- Joined: Wed Oct 18, 2006 3:45 am
- Libera.chat IRC: [com]buster
- Location: On the balcony, where I can actually keep 1½m distance
- Contact:
Re: flat binary
Because that issue hardly existed in real mode at all - everyone was using segmentation. People made it a problem by refusing to use it in protected mode as well.As in, Intel didn't think about implementing an EIP-relative addressing mode like most (all?) other CPU designers did
