Expected stack layout of Win32 process

[rdos]

I'm trying to port the Open Watcom debugger to RDOS, but it seems like it uses some kind of aggressive stack-frame walking. It claims a "Stack Overflow" has occurred just as the execution enters main. There are no stack-exceptions occurring in the code, so this cannot be an issue. I instead suspect it walks the stack and finds something strange in the portion of the stack created before control is passed to the program.

When the program is run outside of the debugger, or without symbolic information, it works just fine.

Is there a document describing the layout of the Win32 stack before entry to an application? Or have compiler constructors reverse-engineered the contents because of the lack of documentation?

[ru2aqare]

I'm trying to port the Open Watcom debugger to RDOS, but it seems like it uses some kind of aggressive stack-frame walking. It claims a "Stack Overflow" has occurred just as the execution enters main. There are no stack-exceptions occurring in the code, so this cannot be an issue. I instead suspect it walks the stack and finds something strange in the portion of the stack created before control is passed to the program.

When the program is run outside of the debugger, or without symbolic information, it works just fine.

Is there a document describing the layout of the Win32 stack before entry to an application? Or have compiler constructors reverse-engineered the contents because of the lack of documentation?

As far as I know, the stack layout on Win32 is not that different from any stack. Initially, some amount of stack (usually 64K) is allocated, and the last page is marked as a guard page. When this page is touched, it raises a page fault, which gets translated into a GUARD_PAGE_VIOLATION exception, which in turn makes Win32 allocate another page for the stack, and mark this new page as the guard page. This way, the stack can grow to a limit (stored in the PE file header). Also, one of the things that gets done before your application receives control is that a SEH frame is pushed on the stack, which allows uncaught SEH exceptions to be handled.

Hope this helps.

[qw]

Initially, some amount of stack (usually 64K) is allocated, and the last page is marked as a guard page. When this page is touched, it raises a page fault, which gets translated into a GUARD_PAGE_VIOLATION exception, which in turn makes Win32 allocate another page for the stack, and mark this new page as the guard page.

IIRC this only works one page at the time, so you can't allocate more than 4096 B at a time.

[AJ]

Hi,

IIRC, you can allocate more than 2k on the stack, using alloca(bytes). There is a newer version of this (_malloca(bytes)), but in the case of _malloca, the memory is allocated on the heap if it exceeds a certain size, whereas alloca guarantees allocation on the stack (if enough memory exists for the allocation).

Cheers,
Adam

[Edit: I guess GCC uses alloca for this purpose - to allocate large areas of stack memory beyond guard page boundaries, because attempting to compile on Cygwin without a Cross-Compiler will sometimes try to bring in references to alloca]

[qw]

Yes, alloca() takes care of allocating stack space in 4 K chunks.