Difference between managing x86 stack and process addresses?

[MilkyGirl]

As far as I know in x86 there are certain trigger instructions in Real Mode that automatically push data on the stack, which is 64 kilobytes if I'm not incorrect.

My main confusion is simple:

If data goes on to the stack, and you must make sure the stack doesn't overflow, and each stack has 64K segments, and there's only one stack, how can you manage the stack between different processes in a multitasking kernel, and the physical address range per thread of execution synonymously?

The hardware stack's use is unknown exactly from my understanding, although I don't know if a "call stack" counts as the same thing, a call stack would be useful for subroutines in a high-level language and program arguments, return "types" and the like(though Assembly languages can have this stuff as well).

But is there something I have wrong here? Is the stack accessed and must be managed constantly so it doesn't overflow while still synonymously maintaining a series of physical addresses ranges per thread/process?

ANY clarification will make my day and anyone else who is interested in this mild confusion.

PS: I am shooting for a DOS-like OS, just with multitasking time-sliced capabilities to run concurrent threads, but I don't really understand this issue with the stack, how and why it's accessed, how it'd impact time-sliced threads and memory ranges, and how it would be managed between multiple execution programs.

[Combuster]

and there's only one stack

Nope. Every task needs a stack of its own. In practice you can borrow the stack of some existing task if you leave it exactly as it was before it's original owner gets it back - which still essentially means having your own stack.

[MilkyGirl]

So adding that in to account, how can my confusion between stack management per process/thread and time-slicing be accomplished in the best, most reasonable approach?

BTW, what do you mean by "owner" of a stack exactly?

Sorry, I'm not that noob-ish, I just don't know all the lingo of this stuff.

[Brendan]

Hi,

As far as I know in x86 there are certain trigger instructions in Real Mode that automatically push data on the stack, which is 64 kilobytes if I'm not incorrect.

In real mode you'd have a maximum of 64 KiB for a stack; but a program could use 4 KiB instead (only using part of an entire segment), or could maybe use multiple stacks (e.g. a huge ugly/complex library that switches to its own stack, does stuff, then switches back to the original stack and returns).

Of course in real mode most software doesn't need much stack (e.g. 4 KiB of probably plenty); partly because real mode is too limited to do anything complex in the first place.

If data goes on to the stack, and you must make sure the stack doesn't overflow, and each stack has 64K segments, and there's only one stack, how can you manage the stack between different processes in a multitasking kernel, and the physical address range per thread of execution synonymously?

Each process (or each thread in each process) would have its own stack. When you switch from one thread or process to another, you'd also switch stacks.

The hardware stack's use is unknown exactly from my understanding, although I don't know if a "call stack" counts as the same thing, a call stack would be useful for subroutines in a high-level language and program arguments, return "types" and the like(though Assembly languages can have this stuff as well).

The hardware stack is just like a normal data segment - the only thing "special" about it is that the CPU has built-in instructions for pushing and popping data that uses SP. If you wanted you could use "fs:di" as a stack instead and it wouldn't make much difference, except you'd need to write your own push/pop code (e.g. "mov [es:di],1234; sub di,2" instead of "push 1234").

PS: I am shooting for a DOS-like OS...

I hope your aim is good - the more DOS-like OSs that get shot the better!


Cheers,

Brendan

[Combuster]

how can my confusion be accomplished

In case you weren't there yet:
Double, double toil and trouble
Fire burn and cauldron bubble?



Anyway, your thoughts appear to be sufficiently scattered that I think you're best off reading a tutorial at this point and get a feeling for all the basics before you get your hands on something as complicated as this - it's definitely not something you pull off in a moment's notice.

[MilkyGirl]

So every process has its own stack?

How do you figure this exactly?

The "stack" would reside in memory, but when I load code from memory, would that already be on the stack?

If not, where would the stack start and end, and where would the stack reside that's not where my bootloader code would be?

Or is the stack briefly some scheme to give the illusion of data in memory? If so, why use it, since all data can theoretically work without it, unless it's hardware specified, and in that case we go back to the drawing board.

I don't get the real purpose, understanding or effectiveness of it really much at all honestly, even reading 500 pages of the "Art of Assembly" guide.

@Combuster

That tutorial is really nothing helping me with the questions I've presented here; if anything it's adding more unneeded work.

I have tried the "learn more" approach. Unfortunately, if I spend too much time learning I end up forgetting more and having to start again. It doesn't solve the issue, and Assembly language, the microprocessor architecture, and the like are extremely confusing when you take in to account every single possibility and use, even being knowledgeable on programming for years and studying this subject.

I would like to KNOW what I'm doing, not just shoot without full knowledge on the target and purpose.

[tom9876543]

I would suggest you need to write some assembly PROGRAMS before trying to write an entire operating system.

Write some win32 asm programs to do basic things like a notepad replacement.

Write some Linux console programs IN ASSEMBLY.

PS Are there any good tutorials on writing a Linux GUI App in Assembly. I guess it would have to explain how to interface with GTK+ using assembly.

When you understand PROGRAMS, you might have some hope of successfully developing an operating system.

[Brendan]

Hi,

So every process has its own stack?

How do you figure this exactly?

The "stack" would reside in memory, but when I load code from memory, would that already be on the stack?

An extremely quick/simplified crash course in machine architecture:

A computer has RAM (which is like a big array of bytes) and a CPU. The bytes in RAM can only hold numbers, but those numbers could represent anything (characters, colours, etc) and 2 or more bytes can be joined together to store larger numbers (that can also represent anything). Some of the numbers represent "instructions", and (if the CPU executes these numbers) they tell the CPU what to do (add, subtract, compare, etc).

Different parts of the RAM are used for different things. Typically a single program might use one area of RAM for code (instructions for the CPU to execute), another few different areas for data, and another different area for stack space. For 80x86 real mode, the CPU uses segment registers to "know" where these areas are. For example, the CS segment register tells the CPU where code is; DS, ES, FS and GS tell the CPU where different data areas are; and the SS segment register tell the CPU where the stack is. Of course segment registers only say where the start of an area is - software uses offsets from the start of the area to find anything stored within that area. For example, you might store something at offset 123 in a segment that starts at the address 10000 (and the CPU will actually store it at the address 10123). Most software doesn't really need to care where segments start (this is the OS's job) and only need to care about which offsets they've used (e.g. software might store something at offset 123 in some segment, without knowing or caring where that segment starts).

Now; nothing prevents you from having multiple programs in memory at the same time. For example, RAM might have 12 different areas for code, 56 different data areas and 12 different stack areas. When an OS switches from one program to another, it has to store the previous process' state somewhere and load the next process' state from somewhere. This state would include the program's segment registers (which tell the CPU where to find the currently running program's code, data and stack).


Cheers,

Brendan

[tom9876543]

Brendan,
it should be obvious writing long winded explanations of basic concepts is a waste of time.
If Ms MilkyGirl is incapable of writing some assembly programs, then she isn't ready for Operating System development.

[Brendan]

Hi,

it should be obvious writing long winded explanations of basic concepts is a waste of time.
If Ms MilkyGirl is incapable of writing some assembly programs, then she isn't ready for Operating System development.

Nobody is ready for OS development until after they've written an OS, and even then it's debatable. The only really important thing is a desire to learn. However, even with a desire to learn sometimes you simply don't realise that there's a gap in your current knowledge, or you do realise there's a gap but have no idea how large that gap is. My post looks like I intended to teach something about computer architecture, but this is just an illusion. My post is intended to make it easier for someone to become aware of any gap/s that may exist in their knowledge.

It is very difficult to assess another person's knowledge from a couple of forum posts. For all I know MilkyGirl may have been programming in COBOL for the US military for the last 40 years, and have far more experience with programming and large projects than I do, and be offended by my "casting a very wide net" previous post.


Cheers,

Brendan

[egos]

So adding that in to account, how can my confusion between stack management per process/thread and time-slicing be accomplished in the best, most reasonable approach?

What about this approach?

[MilkyGirl]

That approach seems to confuse me more.

I know Assembly, never written a program, but I'm a bit lost with the whole stack, stack frame, where the stack is, when it's triggered, how another one is created, threads and processes per stack, stack management, etc.

I understand some of the problem, I just can't manage all of the issues within.

And face it, good programmer or not, writing an operating system is daunting unless you're a pro with years of experience here, and you actually can understand all of those complicating figures in the microarchitecture, aside from endless and hypocritical terminology across endless aspects of design and actual implementation.

PS: Brendan, I know computer architecture, I've been a programmer. What confuses me is the hardware stack, what I said above, and how all of this works in general practice.

[Combuster]

I know Assembly, never written a program

If you haven't automated, let alone get some hands on with the basics, the advanced topics just take exponentially more effort. Right now it seems to be backfiring like the exploits of a certain Sorcerer's Apprentice.

[MilkyGirl]

So you're telling me I should work more on Assembly with user-level software, like application software on an OS, before taking on the architecture more directly?

[Brendan]

Hi,

PS: Brendan, I know computer architecture, I've been a programmer. What confuses me is the hardware stack, what I said above, and how all of this works in general practice.

The CPU has a segment register (SS) that determines where the start of the "stack segment" is, and another register (SP) that determines where the current "top of stack" is within the stack segment.

When something is pushed something on the stack (e.g. instructions like "PUSH" or "CALL"), the CPU decreases the value in SP and then stores the value being pushed at "SS:SP". In C this would look sort of like this:

Code: Select all

uint16_t ss;
uint16_t sp;
uint8_t physicalMemory[1024*1024]      // 1 MiB of pretend physical memory

PUSH(uint16_t value) {
    unsigned int physicalAddress;

    sp -= 2;               // Subtract 2 because we're storing 2 bytes on the stack

    // Store the value at SS:SP (note: little endian)

    physicalAddress = ss * 16 + sp;
    physicalMemory[physicalAddress] = value & 0xFF;
    physicalMemory[physicalAddress+1] = value >> 8;
}

POP is similar but reversed:

Code: Select all

uint16_t POP(void) {
    unsigned int physicalAddress;

    // Get the value at SS:SP (note: little endian)

    physicalAddress = ss * 16 + sp;
    value = physicalMemory[physicalAddress] | (physicalMemory[physicalAddress+1] << 8);

    sp += 2;               // Add 2 because we've taken 2 bytes from the stack
}

To create a stack you just find/allocate some memory somewhere and set SS and SP. For a real mode "DOS like" OS, you'd probably have a memory allocator that returns a segment, so it'd be like this:

Code: Select all

    ss = alloc_segment(new_stack_size);
    sp = new_stack_size;

In assembly, it might look something like this:

Code: Select all

    mov bx,1234           ;Number of bytes to allocate
    call alloc_segment    ;Allocate some memory (allocated segment left in AX)
    mov ss,ax             ;Set the stack segment to the allocated memory
    mov sp,bx             ;Set the current offset within that segment to the top

Of course the OS would allocate a stack when starting a program, and switch to that program's stack later (when switching from one program to another) rather than switching to the stack immediately after allocating it.

So you're telling me I should work more on Assembly with user-level software, like application software on an OS, before taking on the architecture more directly?

To write an application in assembly you need to learn 2 things - assembly itself; and the API's, etc that the application has to use. Obviously learning 2 things is harder than learning one thing. This is especially true for modern OSs where everything is dynamically linked libraries and over-complicated 64-bit calling conventions. If you can find an ancient copy of MS-DOS (or something similar) then it'd be much easier to write an application for DOS in assembly, but you'd still waste time learning about DOS functions (which increases the time it'd take to learn assembly, but may or may not be beneficial anyway if you're planning to write a "DOS like" OS).

In my opinion, screw that. Learning assembly is learning assembly regardless of where you do it; and learning to use another OS's APIs from assembly is just extra hassle and wasted effort. To minimise wasted effort, write assembly for "bare metal". Bare metal means no OS at all. In this case you still need to learn about the firmware's API (e.g. BIOS functions), but you're going to have to learn that to write a "DOS like" OS anyway and therefore it's not wasted effort.

To get you started, download an assembler (e.g. NASM or something) and start with this:

Code: Select all

    org 0x7C00
    jmp start

;Space for BPB here maybe

start:
    jmp 0x0000:here             ;To load CS
here:
    mov ax,0
    mov ds,ax
    mov es,ax
    cli
    mov ss,ax
    mov sp,0x7C00                ;Setup initial stack
    sti

; Start playing with your own assembly code here!


;This loop just stops the CPU when everything above is finished

stop:
    hlt
    jmp stop

;This is needed by some BIOSs to tell them the code is bootable, and has to be at the end of the first sector

    times ($$-$+0x01FE) int3
    dw 0xAA55

;Code for second sector goes here, when you're ready

When you assemble this you should get a 512-byte binary. Install Bochs and enable its built in debugger; and tell Bochs that your 512-byte binary is a floppy disk and boot it. Learn to set breakpoints (e.g. type "vb 0:0x7c00" in Boch's debugger before letting it run anything) and watch your code being executed by single-stepping and seeing what effect each instruction has on registers, etc.

Once you've got that setup and working, start learning assembly by writing your own code to print one character to the screen (this is only about 3 instructions, but will get you used to using a BIOS function). Once that works, try writing a "print string" routine (learn simple loops), then figure out how to load a second sector from disk. From there you can decide what to learn next yourself.


Cheers,

Brendan