Usermode and processes.

[gerryg400]

I can print to screen before the while loop if I don't re-enable interrupts.

That's very good.

What do you mean to attach them?

I meant to attach them to your post so they can be reviewed.

Use a software interrupt without enabling interrupts? what do you mean?

Don't you use software interrupts for system calls ? You can use the INT 0xnn instruction with interrupts disabled. Remember if you do this you need to modify the DPL to ring 3 in the in the type field of the IDT entry for the software interrupt so that you can call it. This is easy to forget.

I suggest you put a printf and a while (1) loop in your interrupt handler until you confirm you can get back into the kernel.

Do you have 2 stacks ?

Don't worry about other things yet. It will make sense as you implement it.

[eXeCuTeR]

I can print to screen before the while loop if I don't re-enable interrupts.

That's very good.

What do you mean to attach them?

I meant to attach them to your post so they can be reviewed.

Use a software interrupt without enabling interrupts? what do you mean?

Don't you use software interrupts for system calls ? You can use the INT 0xnn instruction with interrupts disabled. Remember if you do this you need to modify the DPL to ring 3 in the in the type field of the IDT entry for the software interrupt so that you can call it. This is easy to forget.

I suggest you put a printf and a while (1) loop in your interrupt handler until you confirm you can get back into the kernel.

Do you have 2 stacks ?

Don't worry about other things yet. It will make sense as you implement it.

Sources:
main.c:

Code: Select all

usermode_switch((unsigned int)malloc(512) + 512, (unsigned int)malloc(512) + 512);

usermode_switch.asm:

Code: Select all

extern tss_update
global usermode_switch
usermode_switch:
cli
mov ax, 0x23
mov ds, ax
mov fs, ax
mov es, ax
mov gs, ax

mov eax, [esp+8] ; kernel stack
push eax
call tss_update
add esp, 4 ; cdecl

mov eax, [esp+4] // user stack
push 0x23
push eax
pushf
pop eax
or eax, 0x200
push eax
push 0x1B
push cont

iret

cont:
jmp $

Sorry for no indention, just copied it from my laptop.

TSS functionalities:

Code: Select all

tss_update(unsigned int new_esp0, unsigned int new_ss0)
{
 tss.esp0 = new_esp0;
 tss.ss0 = new_ss0;
}

[gerryg400]

What about LTR and contents of TSS descriptor.

[eXeCuTeR]

What about LTR and contents of TSS descriptor.

Code: Select all

tss_initialize(unsigned int ss0, unsigned int esp0)
{
 memset(&tss, 0, sizeof(tss_entry_t));

 tss.ss0 = ss0;
 tss.cs = 0xB;
 tss.ds = tss.es = tss.fs = tss.gs = tss.gs = tss.ss = 0x13;
}

in gdt_initialize:
creating gdt entries...
tss_initialize(0x10, get_esp());
gdt_add_gate((unsigned int)&tss, (unsigned int)&tss + sizeof(tss_entry_t), 0xE9, 0xCF);

and:

Code: Select all

tss_install:
 mov ax, 0x2B
 ltr ax
 ret

I'm pretty sure that the kernel esp in TSS is damaged.. (esp0)

[gerryg400]

Code: Select all

gdt_add_gate((unsigned int)&tss, (unsigned int)&tss + sizeof(tss_entry_t), 0xE9, 0xCF);

Is this really correct?

[eXeCuTeR]

Code: Select all

gdt_add_gate((unsigned int)&tss, (unsigned int)&tss + sizeof(tss_entry_t), 0xE9, 0xCF);

Is this really correct?

Hey, I entered usermode! thanks mate!
Should int 0x3 or any other int 0xN where N is 0-32 work in usermode? or just int 0x80 (my system call)
Now, when dealing with processes, I would probably want after entering usermode to start spawning new processes.
How do you suggest of doing it (spawning them - creating new processes). implement fork(), after entering usermode, redirect it to some function that forks processes?

[gerryg400]

Hey, I entered usermode! thanks mate!

So interrupts are working ? What was wrong ?

[eXeCuTeR]

Hey, I entered usermode! thanks mate!

So interrupts are working ? What was wrong ?

Kernel stack wasn't appropriate as I suspected.
Thank you very much, you've been really helpful.

BTW: I've heard that each process has it's own kernel stack, is it true? I mean, I can see why: no stack corruption, for example, if PIT comes up when keyboard is still processing, but I'm just not sure if the PIC will keep on sending interrupts unless it has received all EOCs (or it just applies for a single IRQ line and not entire PIC)

One more thing..what should be the size of an appropriate kernel stack for each process (or for all process if the answer to the question above is 'no')?

[Candy]

Yes, each thread has its own kernel stack.

If you didn't, consider this scenario:

Create process with two threads. One calls the syscall Sleep(1). The second waits for 0.5 seconds and then overwrites the kernel return address ont he first stack with whatever he likes. Kernel-level access for your application.

If you want a one-thread scenario, make it do some kind of kernel-copy-memory function and point it to somewhere ont he stack. You can't guarantee that'll break it but you can be sure I can place the kernel return address just in the wrong place.

[skyking]

No, all threads do not necessarily have to have its own kernel stack, you could design the kernel so that only one thread (per core) at a time can be in kernel space at a time. Candy's example is to show that the kernel mode stack should be protected from userspace access, but that is an entirely different story.

Linux uses 8KiB kernel mode stacks per process (on arm at least) so I'd guess that would be a good point to start...

[pcmattman]

BTW: I've heard that each process has it's own kernel stack, is it true? I mean, I can see why: no stack corruption, for example, if PIT comes up when keyboard is still processing, but I'm just not sure if the PIC will keep on sending interrupts unless it has received all EOCs (or it just applies for a single IRQ line and not entire PIC)

Some IRQs should be sent the EOI signal as early as possible (edge-triggered IRQs, IIRC) - meaning another IRQ is allowed to come while you're running the IRQ handler. Other IRQs may cause a context switch that allows a ring3 system call to interrupt processing further down the track, meaning you need a kernel stack per thread (we actually have up to 16 kernel stacks per thread, one for each level of nesting, to avoid stack corruption).

For simplicity on every context switch you can update the TSS esp0 value with the new thread's kernel stack, which allows multiple threads to run in kernel space at a time without conflicts between threads and their stacks.

[gerryg400]

No, all threads do not necessarily have to have its own kernel stack, you could design the kernel so that only one thread (per core) at a time can be in kernel space at a time

Remember the kernel stack serves 2 purposes in x86. Firstly to provide a work area for kernel calls and interrupt service routines, but it is also required by the hardware when transitioning between rings. It is sometimes useful to separate the two.

In some kernels, the esp0 in the TSS points into a small (52byte + room for NMI) 'stack' in the process/thread structure. When a system call or interrupt occurs from userspace, ss3, esp3, flags, cs3 and eip3 are pushed into the thread structure by the processor. Then the isr pushes any other regs that need to be saved into the thread structure too. The kernel then switches to the kernel stack which may be per-core, per-thread, per-interrupt or a combination of those.

[eXeCuTeR]

Thank you all, I've got the concept.

No, all threads do not necessarily have to have its own kernel stack, you could design the kernel so that only one thread (per core) at a time can be in kernel space at a time. Candy's example is to show that the kernel mode stack should be protected from userspace access, but that is an entirely different story.

Linux uses 8KiB kernel mode stacks per process (on arm at least) so I'd guess that would be a good point to start...

I guess I'll stick to having a couple of kernel stacks for each process since designing such kernel will probably cost me of performance and will restrict flexibility of multitasking.

(we actually have up to 16 kernel stacks per thread, one for each level of nesting, to avoid stack corruption).

Level of nesting? what do you mean?
BTW, a little correction if I may - you mean, load TSS's esp0 with the *next* process's kernel stack.

In some kernels, the esp0 in the TSS points into a small (52byte + room for NMI) 'stack' in the process/thread structure. When a system call or interrupt occurs from userspace, ss3, esp3, flags, cs3 and eip3 are pushed into the thread structure by the processor. Then the isr pushes any other regs that need to be saved into the thread structure too. The kernel then switches to the kernel stack which may be per-core, per-thread, per-interrupt or a combination of those.

I couldn't fully figure what you meant in this second paragraph. Could you rephrase shortly please? Just can't find the logic.

[gerryg400]

My thread context looks like this for 64bit but 32 bit is similar.

Code: Select all

typedef struct kobj_thread_t {

	klist_chain_t threadchain;  /* This is 2 pointers == 16 bytes */

	/* START of saved thread context - stack grows down so the context is
	 * filled from oldss to r15. There must be an even number of registers
	 * to satisfy the 16 byte alignment thing.
	 */

	/* These 15 regs are put on the stack by code in kentry.S */
	uint64_t r15;
	uint64_t r14;
	uint64_t r13;
	uint64_t r12;
	uint64_t rbp;
	uint64_t rbx;

	uint64_t r11;
	uint64_t r10;
	uint64_t rax;
	uint64_t r9;
	uint64_t r8;
	uint64_t rcx;
	uint64_t rdx;
	uint64_t rsi;
	uint64_t rdi;

	/* These 5 are pushed by the processor when an int or trap occurs */
	uint64_t rip;
	uint64_t cs;
	uint64_t rflags;
	uint64_t oldrsp;
	uint64_t oldss;

	/* END of thread context */

	/* The address of stk0top must be in Tss.ESP0 when we restart a thread
	 * NOTE: This member must be offset from the beginning of the thread
	 * structure by a multiple of 16 bytes
	 */
	uint64_t stk0top;

When I resume a thread I do a

Code: Select all

		tss[core_id]->esp0 = &curr_thread[core_id]->stk0top
		rsp = &curr_thread[core_id]->r15  <<== TYPO: used to say r11
		pop everything
		iretq

When an interrupt or syscall happens the stack is switched to the stk0top member and stuff gets pushed into the thread structure by the processor and then the isr code. Everything to do with the thread is thus saved in the thread structure.
Then the kernel switches to an appropriate kernel stack to do the actual work.

EDIT: Fixed typo

[pcmattman]

(we actually have up to 16 kernel stacks per thread, one for each level of nesting, to avoid stack corruption).

Level of nesting? what do you mean?

Slightly difficult concept to explain...

Basically, when an event is sent to a thread, that thread is automatically interrupted. This is sort of like an IRQ firing, except local to a thread (events are, essentially, a form of IPC - this is not entirely true, but simplifies the explanation). Because the thread can be interrupted at any time, its context must be saved somewhere. We allow threads to be interrupted even while handling events - so we can't just have a dedicated "event stack".

When an event is dispatched to a thread, the thread's old context is saved onto its current stack. For this example, this is at nesting level 1. The nesting level is incremented to 2, and execution of the event begins. If another event is dispatched to the thread, the context is saved onto the current stack (which is now the second nesting level stack) and the nesting level is incremented to 3.

As each event finishes being handled, the nesting level is reduced by one and the previous context is restored.

The idea is that interrupts and further events (and even ring3/ring0 transitions) don't corrupt the kernel stack for a thread. This is our solution to one of our bigger stack corruption bugs.