Updating the ESP leads to problems

[mrjbom]

Hi.
In my kernel, I try to update the ESP, but after the update works, the variables break, and the return statement doesn't work either.
I want to allocate one page per 4096 bytes for the stack. to do this, I call kmalloc which returns the available address of the 4096 byte page address.

Code: Select all

uint32_t* stack_pointer_esp = 0x0;
int a = 123;
dprintf("--------\n");

//read esp
__asm__ volatile (
	"mov %%esp, %0" : "=r" (stack_pointer_esp)
);
dprintf("esp = 0x%X, a = %i\n", stack_pointer_esp, a);

//alloc 4096 bytes (physical address)
stack_pointer_esp = (uint32_t*)kmalloc(4096);
if ((uint32_t)stack_pointer_esp % 4096) {
	dprintf("stack_pointer_esp address is not aligned!\n");
	kfree((void*)stack_pointer_esp);
	return false;
}
dprintf("stack_pointer_esp = 0x%X\n", stack_pointer_esp);

//set esp
__asm__ volatile (
	"mov %0, %%esp" :: "r" (stack_pointer_esp)
);

//read esp
__asm__ volatile (
	"mov %%esp, %0" : "=r" (stack_pointer_esp)
);

dprintf("1. esp = 0x%X, a = %i\n", stack_pointer_esp, a);
dprintf("2. esp = 0x%X, a = %i\n", stack_pointer_esp, a);
__asm__ volatile (
	"mov %%esp, %0" : "=r" (stack_pointer_esp)
);
dprintf("3 (after read). esp = 0x%X, a = %i\n", stack_pointer_esp, a);
dprintf("4. esp = 0x%X, a = %i\n", stack_pointer_esp, a);

Code: Select all

--------
esp = 0x1147ba0, a = 123
stack_pointer_esp = 0x1148000
1. esp = 0x1148000, a = 123
2. esp = 0x1c, a = 28
3 (after read). esp = 0x1148000, a = 22
4. esp = 0x28, a = 40
--------

The first call to dprintf works fine, but then the variables are reset. This is not a dprintf problem because the debugger also resets.
The register itself is not reset, this can be seen on the 3rd and 4th printing.
The variable a constantly changes its value, it is not clear why.
Why is this happening? If I don't change the ESP, then everything works as it should.

[iansjack]

Are you serious? You alter the stack pointer and then wonder why variables and return don't work.

The best advice I can give is for you to put OS development on hold, find a good book or tutorial on assembler programming, and learn how the computer works at that level. You could usefully experiment with simple user programs. Pay particular attention to how variables are stored - both global and local - how function calls and interrupts work, and the roles of the esp and who registers. You'll also need to familiarise yourself with the way that parameters are passed to functions ( I should really say ways, as there's more than one possible way of doing this).

An understanding of the processor is essential to OS development.

[mrjbom]

Are you serious? You alter the stack pointer and then wonder why variables and return don't work.

The best advice I can give is for you to put OS development on hold, find a good book or tutorial on assembler programming, and learn how the computer works at that level. You could usefully experiment with simple user programs. Pay particular attention to how variables are stored - both global and local - how function calls and interrupts work, and the roles of the esp and who registers. You'll also need to familiarise yourself with the way that parameters are passed to functions ( I should really say ways, as there's more than one possible way of doing this).

An understanding of the processor is essential to OS development.

I was guided by this wiki article.
It's clearly written "In protected mode you set up the stack by just moving a new pointer value into the ESP register"

What did I get wrong?

[PeterX]

Are you serious? You alter the stack pointer and then wonder why variables and return don't work.

The best advice I can give is for you to put OS development on hold, find a good book or tutorial on assembler programming, and learn how the computer works at that level. You could usefully experiment with simple user programs. Pay particular attention to how variables are stored - both global and local - how function calls and interrupts work, and the roles of the esp and who registers. You'll also need to familiarise yourself with the way that parameters are passed to functions ( I should really say ways, as there's more than one possible way of doing this).

An understanding of the processor is essential to OS development.

I was guided by this wiki article.
It's clearly written "In protected mode you set up the stack by just moving a new pointer value into the ESP register"

What did I get wrong?

Well the return went wrong! RET means getting the address from stack and jump to it. So if you alter the stack address, the old values on the stack aren't there anymore. So RET will try to jump to some nonsense address. I totally agree to iansjack's advice. Learn assembler and it will pay off big!

And I must say, inline assembly is tough and errorprone. IMHO it is better to write assembler object file(s) and C object file(s) and then link them together. But if you insist on inline assembly, ok.

Greetings
Peter

[mrjbom]

I will definitely start reading some book about the 8086 processor, this is really my weakness.
But I still don't understand why wiki suggests changing the esp.

[PeterX]

Actually what it says is in other words: IF you have to change the stack in pmode, then do it by changing the ESP. But you don't understand yet what changing to pmode means. It means:
The (real mode) bootloader changes to pmode and then immediatly jumps to the pmode kernel. Then as a first action the kernel has to setup descriptor tables and certain registers (including stack). EDIT: Then (and not earlier) it is ok to call subroutines.

The OSDev Wiki page about stack (the one you mentioned) is not a good starting point to understand these things. Better read how the change to pmode is done. There are texts about that on the internet.

Greetings
Peter

[bzt]

I will definitely start reading some book about the 8086 processor, this is really my weakness.
But I still don't understand why wiki suggests changing the esp.

Yes, you should. Here are the basics in a nutshell with the relevant search phrases in bold.

Stack is list of words, growing downwards, the top pointed by the ESP register. You can push values into it, and pop the value on the top.

The CPU implicitly uses the stack.

• When a CALL instruction is executed, the CPU pushes the return address to the stack
• When RET instruction reached, it pops the value from the top of the stack and continues execution at that address

So simply by calling "dprintf", your code changed the stack (and decreased ESP).

Function local variables are also stored on the stack. To reliably address those variables, something called Stack Frame is used, which has two parts:

• in the function prologue, as the first thing to do, the EBP register is pushed on the stack, and then it is set to the current ESP. Now you have two words on the stack: the return address and the previous EBP. Then on function return,
• in the function epilogue before the RET instruction, this previous EBP popped.

This code is added automatically by your compiler, which then addresses all function local variables relative to EBP (so it does not use ESP, which is a moving target).

Example

Code: Select all

a.c:
int dprintf()
{
}

int main()
{
    dprintf();
}

Compile and disassemble (sorry, I couldn't find a 32 bit compiler, I'm using 64 bit, so there's RSP, RBP, CALLQ, RETQ instead of ESP, EBP, CALL, RET, but the concept is the same):

Code: Select all

$ gcc a.c -o a
$ objdump -d a
...
0000000000001119 <dprintf>:
    1119:	55                   	push   %rbp             <- function prologue
    111a:	48 89 e5             	mov    %rsp,%rbp

    111d:	90                   	nop                     <- function body

    111e:	5d                   	pop    %rbp             <- function epilogue
    111f:	c3                   	retq                    <- "POP InstPointer"

0000000000001120 <main>:
    1120:	55                   	push   %rbp             <- function prologue
    1121:	48 89 e5             	mov    %rsp,%rbp

    1124:	b8 00 00 00 00       	mov    $0x0,%eax        <- function body
    1129:	e8 eb ff ff ff       	callq  1119 <dprintf>   <- "PUSH InstPointer" + "JMP"
    112e:	b8 00 00 00 00       	mov    $0x0,%eax

    1133:	5d                   	pop    %rbp             <- function epilogue
    1134:	c3                   	retq   
...

As you can see, both "main()" and "dprintf()" has extra instructions to handle the stack frame, and those are push/pop which alter the stack.

This means you can change the stack (actually this is essential in context switching), but you must do that carefully, knowing exactly what's in the stack, so that the code that pushed values onto the old stack can pop the same many values from the new stack.

Cheers,
bzt

[mrjbom]

I will definitely start reading some book about the 8086 processor, this is really my weakness.
But I still don't understand why wiki suggests changing the esp.

Yes, you should. Here are the basics in a nutshell with the relevant search phrases in bold.

Stack is list of words, growing downwards, the top pointed by the ESP register. You can push values into it, and pop the value on the top.

The CPU implicitly uses the stack.

• When a CALL instruction is executed, the CPU pushes the return address to the stack
• When RET instruction reached, it pops the value from the top of the stack and continues execution at that address

So simply by calling "dprintf", your code changed the stack (and decreased ESP).

Function local variables are also stored on the stack. To reliably address those variables, something called Stack Frame is used, which has two parts:

• in the function prologue, as the first thing to do, the EBP register is pushed on the stack, and then it is set to the current ESP. Now you have two words on the stack: the return address and the previous EBP. Then on function return,
• in the function epilogue before the RET instruction, this previous EBP popped.

This code is added automatically by your compiler, which then addresses all function local variables relative to EBP (so it does not use ESP, which is a moving target).

Example

Code: Select all

a.c:
int dprintf()
{
}

int main()
{
    dprintf();
}

Compile and disassemble (sorry, I couldn't find a 32 bit compiler, I'm using 64 bit, so there's RSP, RBP, CALLQ, RETQ instead of ESP, EBP, CALL, RET, but the concept is the same):

Code: Select all

$ gcc a.c -o a
$ objdump -d a
...
0000000000001119 <dprintf>:
    1119:	55                   	push   %rbp             <- function prologue
    111a:	48 89 e5             	mov    %rsp,%rbp

    111d:	90                   	nop                     <- function body

    111e:	5d                   	pop    %rbp             <- function epilogue
    111f:	c3                   	retq                    <- "POP InstPointer"

0000000000001120 <main>:
    1120:	55                   	push   %rbp             <- function prologue
    1121:	48 89 e5             	mov    %rsp,%rbp

    1124:	b8 00 00 00 00       	mov    $0x0,%eax        <- function body
    1129:	e8 eb ff ff ff       	callq  1119 <dprintf>   <- "PUSH InstPointer" + "JMP"
    112e:	b8 00 00 00 00       	mov    $0x0,%eax

    1133:	5d                   	pop    %rbp             <- function epilogue
    1134:	c3                   	retq   
...

As you can see, both "main()" and "dprintf()" has extra instructions to handle the stack frame, and those are push/pop which alter the stack.

This means you can change the stack (actually this is essential in context switching), but you must do that carefully, knowing exactly what's in the stack, so that the code that pushed values onto the old stack can pop the same many values from the new stack.

Cheers,
bzt

Thank you for the detailed answer.
I realized that my esp I didn't care about the data that is stored there and this led to an error.

I change the esp in my bootloader.asm which reserves 2 KB. This works as it should.
However, the problem is that the esp changes depending on where I read it. For example, if I read the esp value in kmain, the value is always the same, and if in a function that is called from kmain(), it is different.
I expected the esp to never change from the moment it was installed.
Why is this happening?

In general, I am trying to make a virtual memory and I need to know the exact address of the beginning of the stack in order to add it to the page table, otherwise, when I try to access the stack after setting up paging, I have a problem.

[PeterX]

However, the problem is that the esp changes depending on where I read it. For example, if I read the esp value in kmain, the value is always the same, and if in a function that is called from kmain(), it is different.

I'm not sure if I understand you right. Of course ESP is different in a sub-function than in main. Or do you mean you read ESP twice in the same function and get different results?

I expected the esp to never change from the moment it was installed.

Could it be you confuse ss with esp?

Greetings
Peter

[iansjack]

You really, really need to go back to basics and learn about the x86 processors and assembly language before you go any further. You are knitting together a mish-mash of advice from here and code from the Internet without the knowledge to understand it.

You need to understand the basics of computers before tackling a project like an OS. And, IMO, people are doing you no favours by providing you with snippets of knowledge.

[mrjbom]

You really, really need to go back to basics and learn about the x86 processors and assembly language before you go any further. You are knitting together a mish-mash of advice from here and code from the Internet without the knowledge to understand it.

You need to understand the basics of computers before tackling a project like an OS. And, IMO, people are doing you no favours by providing you with snippets of knowledge.

I read the literature about it.
That's why I'm asking how the stack works here.

[iansjack]

A very good way of understanding how things work is to single-step through some simple assembler programs. Inspecting the code generated when a simple C program is compiled is also instructive. But neither is a substitute for reading. The Intel manuals, though heavy going, contain the information you need. Also there are a number of more accessible books and Internet courses.

Snippets of code from a site like this, or forum posts, are not a good way to learn the subject.

[PeterX]

You really, really need to go back to basics and learn about the x86 processors and assembly language before you go any further. You are knitting together a mish-mash of advice from here and code from the Internet without the knowledge to understand it.

You need to understand the basics of computers before tackling a project like an OS. And, IMO, people are doing you no favours by providing you with snippets of knowledge.

I read the literature about it.

That's why I'm asking how the stack works here.

Well, then you have read the wrong literature! If you would have read a x86 Assembler book you would know how the stack works!

I think there are good gratis tutorials about x86 assembler on the net. If you really don't know the basics like stack etc. than iansjack is right and tips (like from me) won't help you at all.

Use your favorite search engine and osdev.org's wiki to learn assembler. Then (and only then) learn protected mode. Then come back.

I additionally would advice you to stick to application programming before you go for OS developing. I see from your questions that you already have some technical knowledge. But this seems to lead you to the wrong impression you were prepared for OS dev.

As always: This is just an advice to help you. I have not the right or will, to push you away from OS dev if you want to.

Happy hacking
Peter

[mrjbom]

You really, really need to go back to basics and learn about the x86 processors and assembly language before you go any further. You are knitting together a mish-mash of advice from here and code from the Internet without the knowledge to understand it.

You need to understand the basics of computers before tackling a project like an OS. And, IMO, people are doing you no favours by providing you with snippets of knowledge.

I read the literature about it.

That's why I'm asking how the stack works here.

Well, then you have read the wrong literature! If you would have read a x86 Assembler book you would know how the stack works!

I think there are good gratis tutorials about x86 assembler on the net. If you really don't know the basics like stack etc. than iansjack is right and tips (like from me) won't help you at all.

Use your favorite search engine and osdev.org's wiki to learn assembler. Then (and only then) learn protected mode. Then come back.

I additionally would advice you to stick to application programming before you go for OS developing. I see from your questions that you already have some technical knowledge. But this seems to lead you to the wrong impression you were prepared for OS dev.

As always: This is just an advice to help you. I have not the right or will, to push you away from OS dev if you want to.

Happy hacking
Peter

I believe that I am well versed in the work of the PC.
My only problem is the assembler.
In any case, thank you for your answer.
I will definitely read something about x86 processors.

[iansjack]

You might find this resource a useful starting point: https://www.tutorialspoint.com/assembly ... /index.htm

But I'm afraid that you may be overestimating your current level of knowledge. A stack is a very basic concept in computer programming and almost every modern processor makes use of one. This is not simply a question of learning a particular processor's assembly language and instruction set but is a fundamental concept that applies to almost every computer. You will also need to be familiar with other concepts, such as C structures, function calling conventions (ABIs), linked lists, binary trees, how peripheral devices are addressed, etc., etc.

I'm afraid that OS development is one of the hardest tasks you can undertake in programming; it is absolutely essential that you are completely familiar with the fundamentals before you undertake it.