flat binary

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
stanko51
Member
Member
Posts: 32
Joined: Fri Mar 27, 2009 6:58 pm

flat binary

Post by stanko51 »

Hi, I'm not sure about this, so I'm just asking.

Let's say I have a flat binary, compiled and linked with -oformat=binary. i don't specify anything else to the linker.
Can it be copied anywhere in memory (from HDD for example), and just be executed, either by a asm call or by task switching (providing eip with the address) ?
When tested it works fine, but is there some case the code won't work ?

thank you
User avatar
AndreaOrru
Member
Member
Posts: 50
Joined: Fri Apr 25, 2008 2:50 pm
Location: New York

Re: flat binary

Post by AndreaOrru »

You should use -fPIC if you want to be able to execute it anywhere you like.
Last edited by AndreaOrru on Wed Apr 04, 2012 2:43 am, edited 1 time in total.
Close the world, txEn eht nepO
assembler01
Member
Member
Posts: 25
Joined: Mon Feb 27, 2012 9:46 am

Re: flat binary

Post by assembler01 »

stanko51 wrote:Can it be copied anywhere in memory (from HDD for example), and just be executed, either by a asm call or by task switching (providing eip with the address) ?
Yes.
stanko51 wrote:When tested it works fine, but is there some case the code won't work ?
No.
Talk is cheap, show me the code. - Linus Torvalds
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: flat binary

Post by Brendan »

Hi,
stanko51 wrote:Let's say I have a flat binary, compiled and linked with -oformat=binary. i don't specify anything else to the linker.
Can it be copied anywhere in memory (from HDD for example), and just be executed, either by a asm call or by task switching (providing eip with the address) ?
If you use paging and/or segmentation to make sure the code runs from the virtual address it's linked to expect, then there's no problem regardless of what physical address you loaded the code at. If you don't do either of these things and run the code from a virtual address that it isn't linked to expect; then any instruction that relies on a fixed address will cause problems. This includes simple instructions like "mov esi,helloWorldString" and more complex situations like "call [functionTable + eax*4]".

The "-fPIC" thing tells the compiler not to generate instructions that rely on a fixed address. This could mean forcing the use of "RIP relative" addressing in 64-bit 80x86 code (e.g. "lea rsi,[rip+helloWorldString-$]" instead of "mov rsi,helloWorldString"), or using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
User avatar
Solar
Member
Member
Posts: 7635
Joined: Thu Nov 16, 2006 12:01 pm
Location: Germany
Contact:

Re: flat binary

Post by Solar »

Brendan wrote:...using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
I wonder...

At the age of micro-ops, register renaming and other such stunts, how big is the performance penalty of PIC code on modern x86 / x64 cores?
Every good solution is obvious once you've found it.
Nable
Member
Member
Posts: 453
Joined: Tue Nov 08, 2011 11:35 am

Re: flat binary

Post by Nable »

> mov esi,[rbx+helloWorldString]
My message may be rude but maybe you wanted to say
lea esi, [ebx+helloWorldString]
?

> At the age of micro-ops, register renaming and other such stunts, how big is the performance penalty of PIC code on modern x86 / x64 cores?
x64 has RIP-relative addressing, so, looks like this overhead is very small.
For x86 overhead (IMHO) is significant: although we have shadow registers (register renaming), i don't think that CPUs can easily translate the code to use more registers. Execution planning and register allocation is a difficult task even for compilers, and you expect CPU to transform code on the fly, i don't believe in it.
Overuse of PIC is one of the things that i hate most in *nix systems, windows style of relocations is (IMHO) better than using -fPIC for all dynamic libraries. Although, if your application is often stuck waiting for I/O with slower devices (and that's a common situation for GUI apps, file or web servers, etc) then you'll just never notice this penalty of PIC.
User avatar
turdus
Member
Member
Posts: 496
Joined: Tue Feb 08, 2011 1:58 pm

Re: flat binary

Post by turdus »

Brendan wrote:...or using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
Not necessairly. In protmode you can point code and data segment to the address of the code, and use only es,fs,gs for flat memory access. This implies address must be at paragraph border though, but saving a register worth it I think.
User avatar
bubach
Member
Member
Posts: 1224
Joined: Sat Oct 23, 2004 11:00 pm
Location: Sweden
Contact:

Re: flat binary

Post by bubach »

turdus wrote:
Brendan wrote:...or using ugly work-arounds for 32-bit 80x86 code (e.g. waste a register for remembering the address of the code and do "mov esi,[rbx+helloWorldString]" instead of "mov esi,helloWorldString").
Not necessairly. In protmode you can point code and data segment to the address of the code, and use only es,fs,gs for flat memory access. This implies address must be at paragraph border though, but saving a register worth it I think.
yeah, thats why he said this just a few paragraphs above:
Brendan wrote:If you use paging and/or segmentation to make sure the code runs from the virtual address it's linked to expect, then there's no problem regardless of what physical address you loaded the code at.
"Simplicity is the ultimate sophistication."
http://bos.asmhackers.net/ - GitHub
User avatar
Combuster
Member
Member
Posts: 9301
Joined: Wed Oct 18, 2006 3:45 am
Libera.chat IRC: [com]buster
Location: On the balcony, where I can actually keep 1½m distance
Contact:

Re: flat binary

Post by Combuster »

Solar wrote:At the age of micro-ops, register renaming and other such stunts, how big is the performance penalty of PIC code on modern x86 / x64 cores?
On x86 it is obviously a problem: PIC implies that you lose one out of the six remaining registers for computational tasks. That means in turn that you need memory accesses to swap that storage out, meaning more and longer instructions (with corresponding cache effects and decoder pressure), and more uop pressure on the loadstore and address generation units for the stores to happen. Those same units are also responsible for doing EBX-relative addressing. If that load is added onto code that demands of the same processor components, the resulting performance loss is very noticeable (i.e. more than the percentage of registers lost). The first benchmark I found mentions 6% performance loss in practice. (compared to 0.02% on amd64)


On amd64 the entire register pressure effect is simply gone since you can use RIP addressing.
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]
User avatar
Solar
Member
Member
Posts: 7635
Joined: Thu Nov 16, 2006 12:01 pm
Location: Germany
Contact:

Re: flat binary

Post by Solar »

@ Combuster:

Thanks for the benchmark data. (The theory was clear to me, I just wondered what the real-world impact would be like.)
Nable wrote:Overuse of PIC is one of the things that i hate most in *nix systems, windows style of relocations is (IMHO) better than using -fPIC for all dynamic libraries.
PIC is clearly the superior technology compared to relocations. The thing that sucks isn't UNIX using PIC, it's x86 not supporting it properly. (On a MC680x0, for example, you won't have any performance penalty, and I'd expect the same from equally modern architectures like PPC or ARM. Actually, anything but x86. :twisted:
Every good solution is obvious once you've found it.
Nable
Member
Member
Posts: 453
Joined: Tue Nov 08, 2011 11:35 am

Re: flat binary

Post by Nable »

> you won't have any performance penalty
won't have any performance penalty compared to what?
"no performance penalty" != "penalty is always included because we don't have a way to live without it".

Just a simple example: loading a constant offset of variable to register. Do you feel the difference: loading constant vs calculating address?
Oh, you can say that global variables are evil, then what about offsets of functions? What about switch-cases2jump_table optimization?
Relocations are just applied after loading and then .reloc can be discarded. Ability to map one instance of library to different addresses (in different processes) is not smth that is used very often. I mean that in Win32 we have DLLs loaded at same addresses in all threads and in rare case when dll cannot be mapped at the same address in new process because this process image is linked to a perverted base, then we can relocate either new process image or create new instance of dll, relocated to address suitable for this process.

---
Oh, it seems that we are turning this thread into flood+offtop+holy-war so i'll try to keep silence, i.e. stop filling this thread with info that topicstarter didn't ask for.
User avatar
bluemoon
Member
Member
Posts: 1761
Joined: Wed Dec 01, 2010 3:41 am
Location: Hong Kong

Re: flat binary

Post by bluemoon »

Nable wrote:> you won't have any performance penalty
won't have any performance penalty compared to what?
"no performance penalty" != "penalty is always included because we don't have a way to live without it".
Penalty of using PIC compare to NOT using PIC.
Nable wrote:I mean that in Win32 we have DLLs loaded at same addresses in all threads and in rare case when dll cannot be mapped at the same address in new process because this process image is linked to a perverted base, then we can relocate either new process image or create new instance of dll, relocated to address suitable for this process.
Think about Address Space Layout Randomization. On each different relocation you end up with multiple non-shared pages with different patched offset. That can increase stress on memory and cache.
Nable wrote:Oh, it seems that we are turning this thread into flood+offtop+holy-war
No, PIC are critical for running flat binary, and the concern using it (or not) is relavent.
stanko51
Member
Member
Posts: 32
Joined: Fri Mar 27, 2009 6:58 pm

Re: flat binary

Post by stanko51 »

Ok, so I'm left with the choice between:
using -fPIC when linking, or
using relocatable executable and relocate it at runtime

is this correct ?
User avatar
Solar
Member
Member
Posts: 7635
Joined: Thu Nov 16, 2006 12:01 pm
Location: Germany
Contact:

Re: flat binary

Post by Solar »

Nable wrote:> you won't have any performance penalty
won't have any performance penalty compared to what?
As in, Intel didn't think about implementing an EIP-relative addressing mode like most (all?) other CPU designers did ("PC-relative in case of the MC68k family), which makes PIC code trivial. That's why Unix makes such use of PIC: Because the problem with it is specific to the (lack of) x86 addressing modes.
Every good solution is obvious once you've found it.
User avatar
Combuster
Member
Member
Posts: 9301
Joined: Wed Oct 18, 2006 3:45 am
Libera.chat IRC: [com]buster
Location: On the balcony, where I can actually keep 1½m distance
Contact:

Re: flat binary

Post by Combuster »

As in, Intel didn't think about implementing an EIP-relative addressing mode like most (all?) other CPU designers did
Because that issue hardly existed in real mode at all - everyone was using segmentation. People made it a problem by refusing to use it in protected mode as well.
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]
Post Reply