Always switch Stack Segment to 32 bit after entering Protected Mode.

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
Supertos
Posts: 1
Joined: Sat Jul 20, 2024 1:44 pm

Always switch Stack Segment to 32 bit after entering Protected Mode.

Post by Supertos »

It is not a question but a piece on information for those who may have encountered the same problem as i.
If your software enters Protected Mode you shall not leave Stack Segment in 16bits. The problem is that C language works perfectly fine with passing finite arguments to functions, but as soon as you start using variardic arguments things will break: va_arg will either give garbage or zeroes and would not work as intended.

It is no more than hypothesis, but it may be caused by SS not being equal 0 while C assumes flat memory model. This leads to push and pop instructions work fine but direct access to the stack top breaks somehow.
Octocontrabass
Member
Member
Posts: 5560
Joined: Mon Mar 25, 2013 7:01 pm

Re: Always switch Stack Segment to 32 bit after entering Protected Mode.

Post by Octocontrabass »

Lots of things can go wrong if your stack segment doesn't match your code segment.

Intel says using a real-mode segment in protected mode is undefined behavior, so if you do that, you might have additional problems that can't be explained by nonzero segment bases or operand size mismatches.
rdos
Member
Member
Posts: 3296
Joined: Wed Oct 01, 2008 1:55 pm

Re: Always switch Stack Segment to 32 bit after entering Protected Mode.

Post by rdos »

I don't think I have these problems. My kernel stacks are 32-bit, but the size is typically 4k, and they are not expand down, and so it doesn't matter if code use sp or esp since the higher word of esp is always 0. Unless long mode support is enabled, then the kernel stack must be flat, and ss must be loaded with a flat data selector. In this mode some old code that use sp might break, and stack protection will also be broken.
Post Reply