Confusion from Win32 -> Linux 64-bit assembly

[mwbrown]

Hello. I suppose I should introduce myself as well, considering this is my first post. I'm a hobbyist programmer (I'm going to go to college for a CE/CS degree, however), with C, C++, and Java experience. I've recently taken up Assembly as a new programming language, as a result of a newfound interest in computer engineering

I started off with masm32, got a little too used to the invoke syntax, and then ended up moving to nasm after I found the license for masm32 to be incompatible with my plans. I then learned the Win32 way of calling functions, such as (this is an example of some of the libsdl bindings I worked with)

Code: Select all

push dword SDL_INIT_EVERYTHING
call SDL_Init

push dword 0
push dword 32 ; bits per pixel
push dword 480 ; resolution, x
push dword 640 ; resolution, y
call SDL_SetVideoMode
; etc etc

However, I've noticed this does not work very well on Linux 64-bit. I've tried the push/call method, and it doesn't work at all. All my calls are being executed, but I'm not sending the data in the right manner. In fact, if I try that same code on Nasm 2.00, x86_64, I end up completely freezing my window manager! (I do have SDL_Quit in there, so that's not the problem)

Also, one question about something that's been bugging me for quite a while. I've noticed that .lib files that are normally used in Microsoft programming projects are a lot better at producing small code that's easily debuggable, but the static linking that comes with using libc makes it confusing to find the entry point of an assembled binary. Is there any way to dynamically link to libc?

[Alboin]

This article has always worked for me in the past.

[mwbrown]

This article has always worked for me in the past.

That article mentions the method that I've used with 32-bit programming. It works just fine with the 32-bit ABI, but the 64-bit one is just confusing. For instance, if I try something like this printf test:

Code: Select all

BITS 64

extern puts
extern printf

[section .data]

szText   db 'Test print.',0
szFormat db '%s',10,0

[section .text]

global main

main:

	mov rdi, szText
	call puts ; this call works just fine

	mov rdi, szFormat
	mov rsi, szText
	call printf ; segmentation fault (null pointer somewhere?)

	mov rax, 1 ; syscall -> exit
	mov rbx, 0 ; return code = 0
	int 0x80

it segfaults when the call to printf is executed, even though the NASM manual states this on 64-bit code:

The first six integer arguments (from the left) are passed in RDI, RSI, RDX, RCX, R8, and R9, in that order. Additional integer arguments are passed on the stack. These registers, plus RAX, R10 and R11 are destroyed by function calls, and thus are available for use by the function without saving.

This has got me quite stumped, I must say.

[Combuster]

I did find this:

For calls that may call functions that use varargs or stdargs (prototype-less
calls or calls to functions containing ellipsis (. . . ) in the declaration) %al 15 is used
as hidden argument to specify the number of SSE registers used. The contents of
%al do not need to match exactly the number of registers, but must be an upper
bound on the number of SSE registers used and is in the range 0–8 inclusive.

I can't tell wether this actually solves your problem though.

If that doesn't work, your best bet is to write a function with a printf call in C, compile it and disassemble the results to see how its really done.

[binutils]

http://www.tortall.net/projects/yasm/

it supports AMD64

HTH

[Combuster]

(link to different assembler)

Using YASM will not help the OP understanding how vararg calling conventions actually work

As for yasm, it does not provide anything specific to do method invocation so it will not fix the problem either.

[Dex]

Go here: http://flatassembler.net/examples.php
And get "Linux AMD64 examples"