REP STOS Garbage

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
User avatar
Nessphoro
Member
Member
Posts: 308
Joined: Sat Apr 30, 2011 12:50 am

REP STOS Garbage

Post by Nessphoro »

Hello my friends,

While trying to optimize my memory operations code I've stumbled upon a problem.
I've converted my memory clear function to this

Code: Select all

        __asm volatile("mov %0, %%edi":: "r"(Destination):"%edi");
        __asm volatile("mov %0, %%eax":: "r"(0):"%eax");
        __asm volatile("mov %0, %%ecx":: "r"(Count) :"%ecx");
        __asm volatile("cld");
        __asm volatile("rep stosb");
And since the only system that using this at the moment is ClearScreen, I immediately noticed that my screen is filled with green a's.
Any pointers?

EDIT: XOR EAX, EAX helped, but I still don't know the problem
User avatar
iansjack
Member
Member
Posts: 4711
Joined: Sat Mar 31, 2012 3:07 am
Location: Chichester, UK

Re: REP STOS Garbage

Post by iansjack »

Try looking at the assembler code generated when you compile a function using this inline code. I suspect that you will find that line 2 does not do what you think it does. Do you need an input operand for this line?

If you are having problems with inline assembler the first step is to inspect the generated code.

Edit: You might also want to check whether the value that you put in %eax is preserved by the following instruction.
Last edited by iansjack on Sun Jun 10, 2012 1:50 am, edited 1 time in total.
User avatar
bluemoon
Member
Member
Posts: 1761
Joined: Wed Dec 01, 2010 3:41 am
Location: Hong Kong

Re: REP STOS Garbage

Post by bluemoon »

Reference from GCC-Inline-Assembly-HOWTO.html#ss5.2, you can do:

Code: Select all

int foo = 123;
__asm volatile("rep stosb" : :"a"(0), "c"(sizeof(foo)), "D"(&foo) : "memory");
kprintf ("foo = %d\n", foo );
and GCC already know the input registers are clobbered.
PS. It's general assumption to keep cld cleared in C, and you should do cld at startup and keep it cleared.

Of course, it's way better to just do:

Code: Select all

memset(dest,0,count);
And let the compiler emit suitable code.
User avatar
Nessphoro
Member
Member
Posts: 308
Joined: Sat Apr 30, 2011 12:50 am

Re: REP STOS Garbage

Post by Nessphoro »

Alright, I solved the problem.
As expected the problem was in the second instruction.
I've looked at the disassembled code, and found that EAX was getting trashed as it is used for return variables too.
And moreover, since I used "r" operand for __asm, it patched the value through another register, which created chaos even further.
In the end I just changed "r" to "g", to let the compiler decide.
Post Reply