Confused about context switch

[Octocontrabass]

We need to see your code to answer that question.

[KrotovOSdev]

We need to see your code to answer that question.

Here it is:

Code: Select all

__attribute__ ((interrupt)) void sched_time_handler(struct interrupt_frame* frame) {
    static uint8_t time_passed = 0;
    time_passed += 10;

    prepare_rt_tasks();
    if (time_passed >= STD_TIMESLICE) {
        resched();
        time_passed = 0;
    }

    send_eoi(0);
}

[thewrongchristian]

We need to see your code to answer that question.

Here it is:

Code: Select all

__attribute__ ((interrupt)) void sched_time_handler(struct interrupt_frame* frame) {
    static uint8_t time_passed = 0;
    time_passed += 10;

    prepare_rt_tasks();
    if (time_passed >= STD_TIMESLICE) {
        resched();
        time_passed = 0;
    }

    send_eoi(0);
}

When you pre-empt your current process, you call resched() before you send EOI to the PIC. So you'll never get another timer interrupt (or any other IRQ interrupt, for that matter) until the original process is scheduled again somehow.

[KrotovOSdev]

We need to see your code to answer that question.

Here it is:

Code: Select all

__attribute__ ((interrupt)) void sched_time_handler(struct interrupt_frame* frame) {
    static uint8_t time_passed = 0;
    time_passed += 10;

    prepare_rt_tasks();
    if (time_passed >= STD_TIMESLICE) {
        resched();
        time_passed = 0;
    }

    send_eoi(0);
}

When you pre-empt your current process, you call resched() before you send EOI to the PIC. So you'll never get another timer interrupt (or any other IRQ interrupt, for that matter) until the original process is scheduled again somehow.

You're right. I've moved send_eoi(0) before resched() call, thanks.

[KrotovOSdev]

I've noticed some interesting thing. If I change the function for setting up IDT entry by adding flags parameter, I get 0x6 Invalid opcode exception. It sounds like my code is so unstable

[Octocontrabass]

Without seeing the rest of your code, it's hard to say what could be the problem there.

[KrotovOSdev]

Without seeing the rest of your code, it's hard to say what could be the problem there.

That took more time than it should but here is my code

Code: Select all

//irq.c 
void set_idt_entry(void (*irq_handler), uint8_t entry) {
    uintptr_t irq_handler_address = (uintptr_t)irq_handler;

    idt[entry].offset_low = (uint16_t)(irq_handler_address & 0xFFFF);
    idt[entry].selector = 0x08;  // Селектор сегмента кода
    idt[entry].zero = 0;
    idt[entry].flags = entry < 0x20 ? 0x8F : 0x8E;
    idt[entry].offset_high = (uint16_t)((irq_handler_address >> 16) & 0xFFFF);
}

void load_idt() {
    struct {
        uint16_t idt_size;
        uint32_t idt_address;
    } __attribute__((packed)) idt_t;

    idt_t.idt_size = sizeof(idt_entry_t) * 256 - 1;
    idt_t.idt_address = (unsigned long)idt;

    asm volatile ("lidt %0" : : "m" (idt_t));

    return;
}

//timer.c
__attribute__ ((interrupt)) void sched_time_handler(struct interrupt_frame* frame) {
    static uint8_t time_passed = 0;
    time_passed += 10;

    prepare_rt_tasks();

    send_eoi(0);
    if (time_passed >= STD_TIMESLICE) {
        resched();
        time_passed = 0;
    }
}

//sched.c
struct TaskStruct* now_executed;

void resched() {
    asm volatile("sti");

    scheduler_queue_t* task;

    for (uint8_t i = 255; i >= 0; i--) {
        if (scheduler.queue_task_count[i] == 0) continue;
        task = scheduler.scheduler_queues[i];

        do {
            if (task->task.state == TASK_STATE_WAITING || task->task.state == TASK_STATE_RUNNING) {
                task->task.state = TASK_STATE_RUNNING;
                now_executed->state = TASK_STATE_WAITING;

                switch_context(&(task->task));
                return;
            }

            task = task->next_task_queue;
        } while (task != scheduler.scheduler_queues[i]);
    }

    asm volatile("cli");
}

//task_switch.asm
extern now_executed
extern TSS
global switch_context

switch_context:
    push ebx
    push esi
    push edi
    push ebp
 
    mov edi,[now_executed]    ;edi = address of the previous task's "thread control block"
    mov [edi + TCB.esp],esp         ;Save ESP for previous task's kernel stack in the thread's TCB
 
    ;Load next task's state
 
    mov esi,[esp+(4+1)*4]         ;esi = address of the next task's "thread control block" (parameter passed on stack)
    mov [now_executed],esi    ;Current task's TCB is the next task TCB
 
    mov esp,[esi + TCB.esp]         ;Load ESP for next task's kernel stack from the thread's TCB
    mov eax,[esi + TCB.pgd]         ;eax = address of page directory for next task
    mov ebx,[esi + TCB.esp0]        ;ebx = address for the top of the next task's kernel stack
    mov [TSS + TSS_struc.esp0],ebx   ;Adjust the ESP0 field in the TSS (used by CPU for for CPL=3 -> CPL=0 privilege level changes)
    mov ecx,cr3                   ;ecx = previous task's virtual address space
 
    cmp eax,ecx                   ;Does the virtual address space need to being changed?
    je .doneVAS                   ; no, virtual address space is the same, so don't reload it and cause TLB flushes
    mov cr3,eax                   ; yes, load the next task's virtual address space
.doneVAS:
    pop ebp
    pop edi
    pop esi
    pop ebx
 
    ret                           ;Load next task's EIP from its kernel stack

[rdos]

In my experience, after having rewritten the scheduler several times due to poor design, you should separate saving thread state from loading thread state. You should also save ALL registers in your TCB, not just some. Register context should not be saved on the (interrupt) stack as this is begging for problems later when voluntary context switches are added. While the scheduler decides which thread to run next, it should run on a private (per-core) kernel stack. The scheduler should be designed in two parts: The part that handles real-time events, and the part that does load balancing and moving threads between cores.

You also need to design some method so IRQs can wakeup threads, something that complicates things a lot, but which is necessary.

[KrotovOSdev]

In my experience, after having rewritten the scheduler several times due to poor design, you should separate saving thread state from loading thread state. You should also save ALL registers in your TCB, not just some. Register context should not be saved on the (interrupt) stack as this is begging for problems later when voluntary context switches are added. While the scheduler decides which thread to run next, it should run on a private (per-core) kernel stack. The scheduler should be designed in two parts: The part that handles real-time events, and the part that does load balancing and moving threads between cores.

You also need to design some method so IRQs can wakeup threads, something that complicates things a lot, but which is necessary.

IRQ can wakeup thread by setting its state to Waiting in my case but it's not implemented yet.
Do you mean that I have not to use kernel interrupt stack for saving registers? Then do I have to use task stack for saving them?

[rdos]

In my experience, after having rewritten the scheduler several times due to poor design, you should separate saving thread state from loading thread state. You should also save ALL registers in your TCB, not just some. Register context should not be saved on the (interrupt) stack as this is begging for problems later when voluntary context switches are added. While the scheduler decides which thread to run next, it should run on a private (per-core) kernel stack. The scheduler should be designed in two parts: The part that handles real-time events, and the part that does load balancing and moving threads between cores.

You also need to design some method so IRQs can wakeup threads, something that complicates things a lot, but which is necessary.

IRQ can wakeup thread by setting its state to Waiting in my case but it's not implemented yet.
Do you mean that I have not to use kernel interrupt stack for saving registers? Then do I have to use task stack for saving them?

How you do IRQ wakeups depends a bit on if you are fine with running the scheduler will interrupts disabled or not. I want decent interrupt latencies, so I allow the scheduler to be interrupted, but also IRQs to be nested. This requires locks for the scheduler. If an IRQ wants to wakeup a thread, but the IRQ happens while the scheduler is running, then you cannot allow it to manipulate thread lists. I solved this by having a wakeup array of threads (per core). When the scheduler is about to exit it's lock and return to some thread, it will first empty the wakeup array and potentially select one of the threads for execution.

You already have a TCB with current CR3, kernel stack and EIP, so it's easy to save the other registers there too. In the preempt IRQ, you just issue a "reschedule", and let save task & load task handle loading the proper registers. You want to allow kernel threads, and so you shouldn't assume your scheduler will only handle user mode, but also that threads already running on their kernel stack will be preempted. You also want thread to be able to block themselves on resources, and other threads to wake them up. This is different from the preempt interrupt because it doesn't involve a return interrupt stack frame.

[KrotovOSdev]

Sounds nice but I've no idea how to implement that. The only thing I can is add other registers to TCB and maybe lockable resources. So what do I have to change in my scheduler to make kernel threads preemptive? The problem is that the kernel isn't preemptive and therefore the scheduler.

[Octocontrabass]

That took more time than it should but here is my code

Where's the rest of your code?

Code: Select all

    idt[entry].flags = entry < 0x20 ? 0x8F : 0x8E;

Are you sure you want interrupts to remain enabled when an exception occurs? Usually you need to make sure the CPU is in a consistent state in your exception handler before you can enable interrupts.

Code: Select all

__attribute__ ((interrupt)) void sched_time_handler(struct interrupt_frame* frame) {

You need to write interrupt handler entry points in assembly, especially if you want to nest interrupts.

Code: Select all

    asm volatile("sti");
    asm volatile("cli");

That looks backwards. Shouldn't you disable interrupts during a context switch?

Code: Select all

    mov eax,[esi + TCB.pgd]         ;eax = address of page directory for next task
    mov ebx,[esi + TCB.esp0]        ;ebx = address for the top of the next task's kernel stack
    mov [TSS + TSS_struc.esp0],ebx   ;Adjust the ESP0 field in the TSS (used by CPU for for CPL=3 -> CPL=0 privilege level changes)
    mov ecx,cr3                   ;ecx = previous task's virtual address space
 
    cmp eax,ecx                   ;Does the virtual address space need to being changed?
    je .doneVAS                   ; no, virtual address space is the same, so don't reload it and cause TLB flushes
    mov cr3,eax                   ; yes, load the next task's virtual address space
.doneVAS:

This part doesn't need to be written in assembly.

You should also save ALL registers in your TCB, not just some. Register context should not be saved on the (interrupt) stack as this is begging for problems later when voluntary context switches are added.

Aren't all context switches voluntary? Why would saving registers on the stack be a problem? As long as you don't allow context switches inside nested interrupt handlers, you won't have to worry about accessing the nested context.

[KrotovOSdev]

Where's the rest of your code?

I can send it here but it's quite long so I think I shouldn't post it here, should I?

Are you sure you want interrupts to remain enabled when an exception occurs? Usually you need to make sure the CPU is in a consistent state in your exception handler before you can enable interrupts.

Does it mean that I shouldn't use Trap gate for exception handler? Or what do I have to do to make CPU's state consistent.

You need to write interrupt handler entry points in assembly, especially if you want to nest interrupts.

What's the problem with

Code: Select all

__attribute__ ((interrupt))

and how can that stop me from adding nested interrupts? I've also swapped "sti" and "cli" (what a dumb mistake...).

This part doesn't need to be written in assembly.

Ok, I'll rewrite this in C later. I was following https://wiki.osdev.org/Brendan's_Multi-tasking_Tutorial that's why it was written in assembly.

Thank you for your help.

[nullplan]

I can send it here but it's quite long so I think I shouldn't post it here, should I?

The normal answer to this is "github". Or any other source code hosting site.

Does it mean that I shouldn't use Trap gate for exception handler? Or what do I have to do to make CPU's state consistent.

For the first: No. Always use interrupt gates. I don't know what the point of trap gates is, but I see no point in them. Typically it is better to have the CPU disable the interrupt flag, then look at the situation in your interrupt entry code and decide from that whether to re-enable it. This precludes any possibility of a stack overflow if the interrupts are coming too quickly.

For the second question, it depends on your OS. In case of a 64-bit OS, you may need to execute the swapgs instruction if you came from userspace, else you may be in kernel space with a userspace GS.base. That may be a situation you need to look at anyway, because a couple of interrupts cannot be masked out, but you probably don't want to spend the rdmsr to make sure on every interrupt. And on 32-bit kernels, you need to switch the segments over when coming from userspace, but that is something you can skip if you already were in kernel space (segment loads are quite slow, as they have to re-read the GDT).

What's the problem with

Code: Select all

__attribute__ ((interrupt))

and how can that stop me from adding nested interrupts? I've also swapped "sti" and "cli" (what a dumb mistake...).

The attribute is GCC specific and quite underdocumented. I have no idea how the thing is supposed to work on x86, how it can tell between exceptions with error code and those without (do you have to make two different types of register struct?), and how I can get at user registers. I really need to do that for system calls and user space exceptions. These are all uncertainties I don't need in my life, so I just write the thing in assembler, where I can set the rules.

It was also noticed a while back that some compilers will misalign the stackpointer in bad optimization settings, so it is possible that the entry code runs with a misaligned stack pointer, and then you technically need to align the stack pointer and the register struct or else it is undefined behavior in C.

For the nested interrupts: You need to ensure that you have "enough" space left on stack before re-enabling interrupts, which is easy in assembler, but not so easy in C. Maybe that's what Octo meant.

[KrotovOSdev]

I haven't uploaded on github yet, but maybe I should.

For the first: No. Always use interrupt gates. I don't know what the point of trap gates is, but I see no point in them. Typically it is better to have the CPU disable the interrupt flag, then look at the situation in your interrupt entry code and decide from that whether to re-enable it. This precludes any possibility of a stack overflow if the interrupts are coming too quickly.

Ok, I'll change Trap gate to Interrupt gate.

The attribute is GCC specific and quite underdocumented. I have no idea how the thing is supposed to work on x86, how it can tell between exceptions with error code and those without (do you have to make two different types of register struct?), and how I can get at user registers. I really need to do that for system calls and user space exceptions. These are all uncertainties I don't need in my life, so I just write the thing in assembler, where I can set the rules.

So I can't write interrupt handlers on pure C? Then I'll use assembly.

For the nested interrupts: You need to ensure that you have "enough" space left on stack before re-enabling interrupts, which is easy in assembler, but not so easy in C. Maybe that's what Octo meant.

Do I have to check ip esp is larger than my stack limit or what?