ELF Loading: Stack and Arguments

[foliagecanine]

I've recently decided to add ELF program support to my OS (earlier it just loaded flat binaries up to 4MiB!).
I've managed to get the program to load, but I ran into a problem.

With my flat binary, I just mapped a page at a specific address for the stack, and set the stack pointer to stack_page_address+4091.
However, with ELF, the locations and lengths of the sections can vary.
How do I determine where to allocate the stack? From what I can tell, it should be at the top so it can grow down, but how is the top determined?

Secondly, how are arguments and environment variables passed to the executable?

Thanks!

[austanss]

Arguments are rather trivial.

When the program is loaded, all you have to do is pass arguments to it the same way you would with any other function.

[foliagecanine]

So literally just put the char ** on the stack?
Do the actual char arrays get put in the stack as well?

[PeterX]

So literally just put the char ** on the stack?
Do the actual char arrays get put in the stack as well?

Yes, since your OS is 32bit the stackframe is used.
No, the data is somewhere in a memory area that the OS makes available for the application.

Greetings
Peter

[iansjack]

It's not quite as simple as the above answers would imply.

For starters, you are not actually calling a function, so there is no stack frame. What you need to do is to construct a stack that looks as though your program was called as a function. This stack will, obviously, be in the memory space of the new process, not the one that is creating the new process. So you need to allocate memory for the stack in the new processe's memory map, then set up the stack frame. Then you can - in the 32-bit case - push the appropriate values for argc and argv. In the 64-bit case you would load these values into the appropriate registers.

argv will need to point at an array of arguments again in the memory map of the new process. This means that your program loader needs to copy this array from the calling process to the new process (or you could use some sort os shared memory, but I'm not convinced that this is a good idea). Environment variables are normally inherited from the parent process, so you just need to copy the array from the data space one process to the other.

Once this has been done you can jump to the main() function of your new process and it looks as if main() has been called as a function. The work required for all this is normally split between the program loader and the initialization code that is liked into every user program (which you will need to write for your OS). You should also have some code that runs after you return from main() to ensure that the exit() call is made even if the user program contained so explicit return statement.

As for where to place the stack, put it at the top of user memory. Assuming that you are using paging (why wouldn't you), it doesn't matter how big the data segment is. The stack can always reside at the same address - the top of all available user memory.

[thewrongchristian]

I've recently decided to add ELF program support to my OS (earlier it just loaded flat binaries up to 4MiB!).
I've managed to get the program to load, but I ran into a problem.

With my flat binary, I just mapped a page at a specific address for the stack, and set the stack pointer to stack_page_address+4091.
However, with ELF, the locations and lengths of the sections can vary.
How do I determine where to allocate the stack? From what I can tell, it should be at the top so it can grow down, but how is the top determined?

You're in control of the address space, so you can decide to put it wherever you choose. As it is a grow down stack, it might make sense to put it right at the top of user memory. It's not defined in ELF, though there appears to be a GNU extension which I think can set things like no-exec flags on the stack, but I haven't investigated that.

My own personal kernel puts the stack between the bottom of the text segment and the first null mapped page, which gives me about 16MB is space for the user stack.

Secondly, how are arguments and environment variables passed to the executable?

Thanks!

You can pass all your environment and arguments on the stack, add string data to the stack, and keeping track of where those strings get copied in. Finally, once you've copied all your string arguments, copy in the pointer arrays that represent argv and envp, then finally push the locations of the envp and argv array, then the size of argv array as argc.

My code looks like this:

Code: Select all

                void * stacktop = ......;
                /* Copy argv and envp */
                for(int i=0; i<argc; i++) {
                        targv[i] = stacktop = arch_user_stack_pushstr(stacktop, targv[i]);
                }
                for(int i=0; i<envc; i++) {
                        tenvp[i] = stacktop = arch_user_stack_pushstr(stacktop, tenvp[i]);
                }
                tenvp = stacktop = arch_user_stack_mempcy(stacktop, tenvp, (1+envc) * sizeof(*tenvp));
                targv = stacktop = arch_user_stack_mempcy(stacktop, targv, (1+argc) * sizeof(*targv));

                stacktop = arch_user_stack_mempcy(stacktop, &argc, sizeof(argc));

Here, functions like arch_user_stack_mempcy() push arbitrary data to the stack below stacktop, and return the new stacktop that points to that copied data.

At this point, the user stack is set up.

[Octocontrabass]

Secondly, how are arguments and environment variables passed to the executable?

In your OS, you can do it however you like, but the System V i386 and AMD64 psABI both say all of that stuff goes on the stack. (Yes, even though function arguments are passed in registers in the AMD64 psABI, the program loader puts the program arguments on the stack.)

You can find the latest revisions of the i386 and AMD64 psABI here, under the CI/CD job artifacts.

[xeyes]

How do I determine where to allocate the stack?
Thanks!

The standard way is to use the generic virtual space allocator to allocate stack space.

If you don't have one (like me), a step stone method can be manually assign a fixed area and put a tiered structure there. Like assign 512MB, which can hold 128 standard threads of 2MB each, and leave the other 256MB for maybe 4 huge stack threads that can each take 64MB, or 2x128MB. This is also more predictable during debugging, you can easily tell whether it is the main thread or the 3rd one and whether it is running up against the limit just from the pointers/stack address used.

The elf loader can check and abort if the executable wants to load within this fixed area and you can move it around as needed.

Secondly, how are arguments and environment variables passed to the executable?
Thanks!

There are 2 parts:

1. Transportation
Many ways. You can manipulate the stack like thewrongchristian did according to "calling convention" to fake it. You can also make special arrangements to do register passing (like how a multiboot boot loader passes pointer via EBX). You can even per-arrange a jump pointer at address 0x0 which is unlikely to be otherwise used. etc

2. Presentation
If you are just running your programs, the transportation can be the presentation.

If you run "standard" C programs, they expect to be started by a call to

Code: Select all

int main (int argc, char *argv[]) 

But main doesn't need to be the actual user point entry. Which is probably the ostensibly parameter-less _start, you can and probably need to implement it. _start can get the parameters using the same transportation just like your own programs, and then arrange them into argc and argv before making the call to main.

This is just the high level idea. There can be various unexpected caveats though, such as the interesting "optimized out" bug below