Kernel hangs switching page directory during task switch

[SlayterDev]

I'm implementing multitasking into my kernel and most everything is going ok until it is time to switch tasks. From what I can tell, it seems to hang when we switch page directories. Here is the task switch function:

Code: Select all

void task_switch() {
	if (!current_task)
		return;

	kprintf(K_DEBUG, "Switching task!\n");
	u32int esp, ebp, eip;
	asm volatile("mov %%esp, %0" : "=r"(esp));
	asm volatile("mov %%ebp, %0" : "=r"(ebp));

	eip = read_eip();
	if (eip == 0x12345)
		return;

	current_task->eip = eip;
	current_task->esp = esp;
	current_task->ebp = ebp;

	current_task = current_task->next;
	if (!current_task) current_task = ready_queue;

	esp = current_task->esp;
	ebp = current_task->ebp;

	//kprintf(K_DEBUG, "Lets do the do!\n");
	asm volatile(" \
		cli; \
		mov %0, %%ebx; \
		mov %1, %%esp; \
		mov %2, %%ebp; \
		mov %3, %%cr3; \
		mov $0x12345, %%eax; \
		sti; \
		jmp *%%ebx; \
		" : : "r"(eip), "r"(esp), "r"(ebp), "r"((physical_addr)&cur_directory->m_entries)
		  : "%ebx", "%esp", "%eax");
}

I'm using the virual memory model from the Brokenthorn Entertainment tutorial series. So I had to roll my own clone directory function. This may be where the issue is but I'm not sure how to tell. Here is the code from that section:

Code: Select all

ptable *virt_clone_table(pd_entry *src) {
	ptable *table = (ptable *)mem_alloc_block();
	ptable *srcTable = (ptable *)PAGE_GET_PHYSICAL_ADDRESS(src);

	memset(table, 0, sizeof(ptable));

	for (int i = 0; i < 1024; i++) {
		if (!srcTable->m_entries[i])
			continue;

		pt_entry page = 0;

		if (pt_entry_is_present(srcTable->m_entries[i])) pt_entry_add_attrib(&page, PTE_PRESENT);
		if (pt_entry_is_writable(srcTable->m_entries[i])) pt_entry_add_attrib(&page, PTE_WRITABLE);
		if (pt_entry_is_user(srcTable->m_entries[i])) pt_entry_add_attrib(&page, PTE_USER);
		if (pt_entry_is_accessed(srcTable->m_entries[i])) pt_entry_add_attrib(&page, PTE_ACCESSED);
		if (pt_entry_is_dirty(srcTable->m_entries[i])) pt_entry_add_attrib(&page, PTE_DIRTY);

		pt_entry_set_frame(&page, pt_entry_pfn(srcTable->m_entries[i]));
	}

	return table;
}

pdirectory *virt_clone_directory(pdirectory *src) {
	pdirectory *dir = (pdirectory *)mem_alloc_block();

	memset(dir, 0, sizeof(pdirectory));

	u32int phys = (physical_addr)&dir->m_entries;

	for (int i = 0; i < 1024; i++) {
		if (!src->m_entries[i])
			continue;

		if (kernel_directory->m_entries[i] == src->m_entries[i]) {
			dir->m_entries[i] = src->m_entries[i];
		} else {
			// copy the table
			ptable *table = virt_clone_table(&src->m_entries[i]);

			pd_entry *entry = (pd_entry *)&dir->m_entries[i];
			pd_entry *srcTable = (pd_entry *)&src->m_entries[i];

			if (pd_entry_is_present(src->m_entries[i]))  pd_entry_add_attrib(srcTable, PDE_PRESENT);
			if (pd_entry_is_writable(src->m_entries[i])) pd_entry_add_attrib(srcTable, PDE_WRITABLE);
			if (pd_entry_is_user(src->m_entries[i])) 	  pd_entry_add_attrib(srcTable, PDE_USER);
			if (pd_entry_is_accessed(src->m_entries[i])) pd_entry_add_attrib(srcTable, PDE_ACCESSED);
			if (pd_entry_is_dirty(src->m_entries[i]))    pd_entry_add_attrib(srcTable, PDE_DIRTY);

			//dir->m_entries[i] = src->m_entries[i];
			pd_entry_set_frame(entry, (physical_addr)table);
		}
	}

	return dir;
}

When it switches directories, it doesn't cause any exceptions or faults. Not even a page fault. I'd appreciate some device on what might be the cause or where to begin to look deeper.

Thanks

[iansjack]

Well, if it just hangs then it sounds like it's stuck in a loop or else has hit a "hlt" instruction. Run it under a debugger; then interrupt it when it hangs and see what's happening.

[KemyLand]

These are my observations:
1- *DON'T* do this in C code! The compiler can optimize important things. Use pure assembly instead
2- You're not saving ECX, EDX, EDI, ESI, and EFLAGS. You *really* should.
3- You're not restoring registers.
4- The deadlock might be due to a compiler optimization. Again, use assembly.
5- Why do you move 0x12345 to EAX? Debugging maybe?
6- Why are you passing "((physical_addr)&cur_directory->m_entries)" as your Page Directory? You should pass the *next* paging directory instead
7- Another inline assembly problem: You're using the "r" constraint to pass the values. What if GCC decided to store something in EBX? Then you override it. Next with all the other ones! I don't even know how the registers can end up. Solution: Assembly
8- Please see the OSDev Wiki article on the matter so you can understand what you should (and shouldn't do). This one is also useful.

Please tell us if you have more problems. People in this forum likes to help others . I'll edit the first article soon, as it uses C (DON'T DO IT!).

[SlayterDev]

Oh **** I'm dumb haha I didn't even realize I was using the same page directory. Ok well now I updated the code to use the new page directory but now I get a triple fault. No exceptions triggered, it just straight up triple faults. This is before trying to re do this in assembly, just wanted to see if the quick fix would work. Any thoughts? (besides using assembly, i'm working on it lol)

EDIT: Well actually thats probably because interrupts are disabled haha. Would that mean the page directory is messed up? Thats what I'm thinking...

[SlayterDev]

Hmmm I think I got the page directory sorted but now I get a General Protection Fault when switching to the new task.

[KemyLand]

Oh **** I'm dumb haha I didn't even realize I was using the same page directory. Ok well now I updated the code to use the new page directory but now I get a triple fault. No exceptions triggered, it just straight up triple faults. This is before trying to re do this in assembly, just wanted to see if the quick fix would work. Any thoughts? (besides using assembly, i'm working on it lol)

EDIT: Well actually thats probably because interrupts are disabled haha. Would that mean the page directory is messed up? Thats what I'm thinking...

And you are thinking well. C is not made to handle this stuff. The register mess disordered all, giving out a little triple fault . Well, remember that if a triple fault happens your double fault handler couldn't be reached. That's good because we have advanced in this, and bad because its a triple fault

OK now to the solution. I will give you some assembly/C code. Please test it because I just baked it and could have any potential bug... The C code just controls the operation

ldpagedir.S

Code: Select all

.text
.globl loadPageDirectory
loadPageDirectory:
push %ebp
movl %esp, %ebp
movl 8(%esp), %eax
movl %eax, %cr3
movl %ebp, %esp
pop %ebp
ret

setpagbit.S

Code: Select all

.text
.globl setPagingBit
setPagingBit:
push %ebp
movl %esp, %ebp
movl 8(%esp), %eax
movl %cr0, %ecx
cmp $0x00, %eax
jne .onSet
.onUnset:
orl $0x00, %ecx
jmp .end
.onSet:
orl $0x01, %ecx
.end:
movl %ecx, %cr0
movl %ebp, %esp
pop %ebp
ret

switch.S

Code: Select all

.text
.globl switchTask
switchTask:
pushal
pushfl
movl %cr3, %eax # Push CR3
push %eax
movl 44(%esp), %eax # The first argument, where to save
movl %ebx, 4(%eax)
movl %ecx, 8(%eax)
movl %edx, 12(%eax)
movl %esi, 16(%eax)
movl %edi, 20(%eax)
movl 36(%esp), %ebx # EAX
movl 40(%esp), %ecx # EIP
movl 20(%esp), %edx # ESP
addl $4, %edx # Remove the return value ;)
movl 16(%esp), %esi # EBP
movl 4(%esp), %edi # EFLAGS
movl %ebx, (%eax)
movl %edx, 24(%eax)
movl %esi, 28(%eax)
movl %ecx, 32(%eax)
movl %edi, 36(%eax)
pop %ebx # CR3
movl %ebx, 40(%eax)
push %ebx # Goodbye again ;)
movl 48(%esp), %eax # Now it is the new object
movl 4(%eax), %ebx # EBX
movl 8(%eax), %ecx # ECX
movl 12(%eax), %edx # EDX
movl 16(%eax), %esi # ESI
movl 20(%eax), %edi # EDI
movl 28(%eax), %ebp # EBP
push %eax
movl 36(%eax), %eax # EFLAGS
push %eax
popfl
pop %eax
movl 24(%eax), %esp # ESP
push %eax
movl 44(%eax), %eax # CR3
movl %eax, %cr3
pop %eax
push %eax
movl 32(%eax), %eax # EIP
xchg (%esp), %eax # We don't have more registers to use as tmp storage
movl (%eax), %eax # EAX
ret # This ends all!

task.h

Code: Select all

#ifndef __TASK_H__
#define __TASK_H__

#include <stdint.h>
extern void initTasking();

typedef struct {
  uint32_t eax, ebx, ecx, edx, esi, edi, esp, ebp, eip, eflags, cr3;
} Registers;

typedef struct Task {
  Registers regs;
  struct Task *next;
} Task;

extern void initTasking();
extern void createTask(Task*, void(*)(), uint32_t, uint32_t*);

extern void preempt(); // Switch task frontend
extern void switchTask(Registers *old, Registers *new); // The asm function

#endif /* __TASK_H__ */

paging.h

Code: Select all

#ifndef __PAGING_H__
#define __PAGING_H__

#include "task.h"

extern void initPaging(Task *task);

extern void loadPageDirectory(uint32_t*); // asm
extern void setPagingBit(uint8_t); // asm

#endif /* __PAGING_H__ */

task.c

Code: Select all

#include "task.h"
#include "paging.h"

static Task *runningTask;
static Task mainTask;
static Task otherTask;

static void otherMain() {
  printk("Hello multitasking world!"); // Not implemented here...
  preempt();
}
	
void initTasking() {
  initPaging(&mainTask);
  asm volatile("pushfl; movl (%%esp), %%eax; movl %%eax, %0; popfl;":"=m"(mainTask.regs.eflags)::"%eax");
  
  createTask(&otherTask, otherMain, mainTask.regs.eflags, (uint32_t*)mainTask.regs.cr3);
  
  mainTask.next = &otherTask;
  otherTask.next = &mainTask;
  
  runningTask = &mainTask;
}

void createTask(Task *task, void (*main)(), uint32_t flags, uint32_t *pagedir) {
  task->regs.eax = 0;
  task->regs.ebx = 0;
  task->regs.ecx = 0;
  task->regs.edx = 0;
  task->regs.esi = 0;
  task->regs.edi = 0;
  task->regs.eflags = flags;
  task->regs.eip = (uint32_t)main;
  task->regs.cr3 = (uint32_t)pagedir;
  task->regs.esp = (uint32_t)allocPage() + 0x1000; // Not implemented here
  task->next = 0;
}

void preempt() {
  Task *last = runningTask;
  runningTask = runningTask->next;
  switchTask(&last->regs, &runningTask->regs);
}

paging.c

Code: Select all

#include <stdint.h>
#include "paging.h"
#include "task.h"

static uint32_t *pagedir;
static uint32_t *pagetab;

void initPaging(Task *main) {
  pagedir = allocPage(); // Not here
  pagetab = allocPage(); // Not here
      
  for(int i = 0; i < 1024; i++)
    pagedir[i] = 0x00000002;
  
  for(int i = 0; i < 1024; i++)
    pagetab[i] = (i * 0x1000) | 3;
  
  pagedir[0] = ((uint32_t)pagetab) | 3;
  
  loadPageDirectory(pagedir);
  setPagingBit(1);
  
  main->regs.cr3 = (uint32_t)pagedir;
}

3rd or 4th Edit: After some major bugs were discovered (thanks, GDB!), this code is tested and is known to work. It'll be added soon to the wiki .

[SlayterDev]

Thanks for the code! When I get a chance to integrate this I will report back with the results!

[SlayterDev]

Looks like that code causes the single step trap to be triggered (Or at least some exception on that line). I'm sure its probably just some small arithmetic error with the eflags. I checked them and before attempting to switch tasks, the eflags are 0x200216 and at the fault they come out as 0x7F53. I would take the time to step through the assembly and see where the error is but alas I must get some sleep. Will check it out in the morning. Thanks for the help!

[KemyLand]

You were completely right. I had two arithmetic mistakes near the end of the function:

This one causes the single step

Code: Select all

movl 40(%eax), %eax

This line causes the code to move saved *CR3* to register *EFLAGS* . It must be

Code: Select all

movl 36(%eax), %eax

Which moves saved *EFLAGS* to register *EFLAGS*

The other one

Code: Select all

movl 40(%eax), %eax # CR3

This line causes the code to move saved *EFLAGS* to register *CR3* . It must be

Code: Select all

movl 44(%eax), %eax # CR3

Which moves saved *CR3* to register *CR3*

If you happened to pass the first bug, the second one would triple fault. I'll edit my answer to repair the bugs. Try it and comment your results!


As a side note, this only works for kernel tasks . The switch to user-mode is several times more complex. In this code, both page directories *must* have their counterpart's stack mapped in the same place.

Tip: A simple way to implement multitasking is to adapt this code to iret instead of ret. This needs even more hack with the stack (and assembly)

[SlayterDev]

Hmmm, the eflags still seem to be messed up. That seemed to change them but still off.

[KemyLand]

I'm laughing so hard to myself in this moment... See this line

Code: Select all

movl (%eax), %eax

As Combuster once said:

What happens to the value of x?

Jeje. The fix is to move that line *just* before the ret. I'll edit (again ) my answer for completeness.

[SlayterDev]

Now we get a triple fault. So different error so at least we're getting somewhere haha. I did have a suspicion about that line but wasn't really sure lol.

[KemyLand]

Now we get a triple fault. So different error so at least we're getting somewhere haha. I did have a suspicion about that line but wasn't really sure lol.

You're right; we're getting something with a triple fault, contrary to what most developers think. Well, the triple fault occurs when a exception ocurred while calling the double fault ISR, which is called when an exception ocurred calling an exception handler . This means only one thing: something paging-related. I shouldn't have coded that assembly when I was falling asleep . Now, it's time to debug (again...) . Let's start by asking you, where does the triple fault occurs?

Edit 1: I mounted the Bare Bones up with this examples with the only purpose of debugging my buggy assembly with GDB. Remember those __attribute__((aligned(4096)))? They cause a major problem. You should use a Page Frame Allocator instead to get them . They made the actual objects to be page-aligned, while the pointers passed are the original addresses . Just adapt it yourself and all will be ok . It's incredible what a debugger can help! But I think there're still some bugs GDB must find

Edit 2: Hi! After some hours of debug with GDB, I've finally managed to make the code work. Please reintegrate all the sources, as I changed some minor but important things (specially in the assembly code). I'm sure it works .

[SlayterDev]

Ok sorry for the absence, I was at a conference all last week. So I implemented the changes you made in the switch task function as well as the stack register in the create_task() function and now it seems to hang when we switch tasks. I should note that I'm using my Virtual Memory manager to get the page directories as It would be a real hassle to mess with how that is currently set up. So the problem could be there but I'm not sure.

[FallenAvatar]

...So the problem could be there but I'm not sure.

Do your own debugging and be sure! Use Bochs or something similar with debuggin enabled (or DEBUG level logging enabled!)

- Monk