return during "int"

[GLneo]

my system call set up works like this:

Code: Select all

_int_handler_set:
    push eax
    push ebx
    push ecx
    push edx
    mov eax, _int_handler
    call eax
    iret

which gets called after int 30h, which in turn calls:

Code: Select all

void int_handler(volatile unsigned int EDX, volatile unsigned int ECX, 
            volatile unsigned int EBX, volatile unsigned int EAX)
{
    if(EAX == 1)
        free((void *)EBX);
    else if(EAX == 2)
        EAX = (unsigned int)malloc(EBX);
    else if(EAX == 3)
        cls();
    else if(EAX == 4)
        putchar((char)EBX);
    else if(EAX == 5)
        putint(EBX);
    else if(EAX == 6)
        puts((unsigned char *)EBX);
    else if(EAX == 7)
        attr((short)EBX);
}

but i think whats wrong is the functions get called and after they do what they do, they "ret" but because it has been after an "int" it should be a "iret" so the sys goes crazy, help???

[Brendan]

Hi,

but i think whats wrong is the functions get called and after they do what they do, they "ret" but because it has been after an "int" it should be a "iret" so the sys goes crazy, help???

The C functions get "CALLed" from within the assembly stub, and therefore should use a "RET" to return to the assembly stub.

The problem is your assembly stub, which pushes a pile of general registers onto the stack that are never popped off again. Try changing to:

Code: Select all

_int_handler_set:
    pushad
    push eax
    push ebx
    push ecx
    push edx
    call _int_handler
    add esp,16
    popad
    iret

Cheers,

Brendan

[Pype.Clicker]

oh, btw, you might want to write free, malloc, etc. into a syscall_table[] and issue

Code: Select all

syscall_table[EAX](EBX,ECX,EDX)

rather than having them in a bunch of "if ... elsif .. elsif ..."

[GLneo]

thx, brendan, that fixed it!

@Pype.Clicker: how would i do that, just have "syscall_table" have an array of pointers to functions?

[Pype.Clicker]

yes, that's roughly the idea ...