Problem running an application in ring 3

[apamment]

Hi there,

I've been trying to run an application on my OS, it loads and runs fine in Ring 0, but when I switch it to Ring 3 it goes wrong. I think my problem may be with my TSS, I am finding lots of conflicting information about how to set that up. I have this:

Code: Select all

#define GDT_TSS SEG_GRAN(1) | SEG_PRES(1) | SEG_PRIV(0) | SEG_SIZE(1) | SEG_CODE_EXA

this defines the flags of the TSS based on http://wiki.osdev.org/GDT_Tutorial#Some ... _life_easy, the base is the location of the TSS and the size is the size of the TSS

It seems to be working, in that I'm not getting any fault 10s, and it successful switches to ring 3, back to ring 0 for a syscall, however this is where things mess up.

the values in the registers are all wrong. mostly 0xffffffff and it page faults trying to access 0xffffffff.

So my other theory is I'm doing something wrong with the stack. When I page fault I print out from ESP to the kstack top i get:

e <- 14, page fault
5 <- error code (page not accessible)
ffffffff <- eip ???
1b <- cs
10296 <- eflags
f0201fc0 <- user esp
23 <- userss

i've been going in circles a couple of days on this, and i don't really have an idea what's going wrong and what information is relevant.

[BASICFreak]

check your IDT entry and exit code, make sure you are not screwing the stack - which is easy to do.

EDIT: Also it is hard to debug without knowing your TSS or IDT

[apamment]

this is my interrupt code

Code: Select all

irq16:
	push byte 0
	push byte 48

	cld
	pushad
	push ds
	push es
	push fs
	push gs

	mov eax, [current_task]
	mov [eax], esp
	
	mov ax, 0x10
	mov ds, ax
	mov es, ax
	mov fs, ax
	mov gs, ax

	push esp

	call irq_handler

	add esp, 4

	pop gs	 
	pop fs
	pop es
	pop ds
	popad

	add esp, 8

	iret

Not sure what you mean by knowing my TSS? This is my struct

Code: Select all

struct tss_t {
  unsigned short backlink, __blh;
  unsigned int esp0;
  unsigned short ss0, __ss0h;
  unsigned int esp1;
  unsigned short ss1, __ss1h;
  unsigned int esp2;
  unsigned short ss2, __ss2h;
  unsigned int cr3;
  unsigned int eip;
  unsigned int eflags;
  unsigned inteax, ecx, edx, ebx;
  unsigned int esp, ebp, esi, edi;
  unsigned short es, __esh;
  unsigned short cs, __csh;
  unsigned short ss, __ssh;
  unsigned short ds, __dsh;
  unsigned short fs, __fsh;
  unsigned short gs, __gsh;
  unsigned short ldt, __ldth;
  unsigned short trace, bitmap;
}__attribute__((packed));

this is how i fill it out

Code: Select all

  memset(system_tss, 0, sizeof(struct tss_t));
  system_tss.cs = 0x8;
  system_tss.ss0 = system_tss.es =  system_tss.ss = system_tss.ds = system_tss.fs = system_tss.gs = 0x10;
  system_tss.cr3 = (long)page_directory;

This is how I set my IDT:

Code: Select all

void idt_set_gate(unsigned char num, unsigned long base, unsigned short sel, unsigned char flags) {
	idt[num].base_lo = base & 0xffff;
	idt[num].sel = sel;
	idt[num].always0 = 0;
	idt[num].flags = flags;
	idt[num].base_hi = base >> 16 & 0xffff;
}

this is what i set for ISRs and IRQs

Code: Select all

void init_isrs() {
    idt_set_gate(0, (unsigned)isr0, 0x08, 0x8E);
    idt_set_gate(1, (unsigned)isr1, 0x08, 0x8E);
    idt_set_gate(2, (unsigned)isr2, 0x08, 0x8E);
    idt_set_gate(3, (unsigned)isr3, 0x08, 0x8E);
    idt_set_gate(4, (unsigned)isr4, 0x08, 0x8E);
    idt_set_gate(5, (unsigned)isr5, 0x08, 0x8E);
    idt_set_gate(6, (unsigned)isr6, 0x08, 0x8E);
    idt_set_gate(7, (unsigned)isr7, 0x08, 0x8E);
    idt_set_gate(8, (unsigned)isr8, 0x08, 0x8E);
    idt_set_gate(9, (unsigned)isr9, 0x08, 0x8E);
    idt_set_gate(10, (unsigned)isr10, 0x08, 0x8E);
    idt_set_gate(11, (unsigned)isr11, 0x08, 0x8E);
    idt_set_gate(12, (unsigned)isr12, 0x08, 0x8E);
    idt_set_gate(13, (unsigned)isr13, 0x08, 0x8E);
    idt_set_gate(14, (unsigned)isr14, 0x08, 0x8E);
    idt_set_gate(15, (unsigned)isr15, 0x08, 0x8E);
    idt_set_gate(16, (unsigned)isr16, 0x08, 0x8E);
    idt_set_gate(17, (unsigned)isr17, 0x08, 0x8E);
    idt_set_gate(18, (unsigned)isr18, 0x08, 0x8E);
    idt_set_gate(19, (unsigned)isr19, 0x08, 0x8E);
    idt_set_gate(20, (unsigned)isr20, 0x08, 0x8E);
    idt_set_gate(21, (unsigned)isr21, 0x08, 0x8E);
    idt_set_gate(22, (unsigned)isr22, 0x08, 0x8E);
    idt_set_gate(23, (unsigned)isr23, 0x08, 0x8E);
    idt_set_gate(24, (unsigned)isr24, 0x08, 0x8E);
    idt_set_gate(25, (unsigned)isr25, 0x08, 0x8E);
    idt_set_gate(26, (unsigned)isr26, 0x08, 0x8E);
    idt_set_gate(27, (unsigned)isr27, 0x08, 0x8E);
    idt_set_gate(28, (unsigned)isr28, 0x08, 0x8E);
    idt_set_gate(29, (unsigned)isr29, 0x08, 0x8E);
    idt_set_gate(30, (unsigned)isr30, 0x08, 0x8E);
    idt_set_gate(31, (unsigned)isr31, 0x08, 0x8E);

}

void init_irqs(void) {
	int i;
	
	irq_remap();

	idt_set_gate(32, (unsigned)irq0, 0x08, 0x8E);
	idt_set_gate(33, (unsigned)irq1, 0x08, 0x8E);
	idt_set_gate(34, (unsigned)irq2, 0x08, 0x8E);
	idt_set_gate(35, (unsigned)irq3, 0x08, 0x8E);
	idt_set_gate(36, (unsigned)irq4, 0x08, 0x8E);
	idt_set_gate(37, (unsigned)irq5, 0x08, 0x8E);
	idt_set_gate(38, (unsigned)irq6, 0x08, 0x8E);
	idt_set_gate(39, (unsigned)irq7, 0x08, 0x8E);
	idt_set_gate(40, (unsigned)irq8, 0x08, 0x8E);
	idt_set_gate(41, (unsigned)irq9, 0x08, 0x8E);
	idt_set_gate(42, (unsigned)irq10, 0x08, 0x8E);
	idt_set_gate(43, (unsigned)irq11, 0x08, 0x8E);
	idt_set_gate(44, (unsigned)irq12, 0x08, 0x8E);
	idt_set_gate(45, (unsigned)irq13, 0x08, 0x8E);
	idt_set_gate(46, (unsigned)irq14, 0x08, 0x8E);
	idt_set_gate(47, (unsigned)irq15, 0x08, 0x8E);
	idt_set_gate(48, (unsigned)irq16, 0x08, 0xEE); // syscall
}

[BASICFreak]

system_tss.esp0 ?

The only fields REQUIRED in a tss is ss0 and esp0.

Without setting esp0, the stack could be anywhere during a ring switch.

Also:
unsigned short backlink, __blh;
unsigned int esp0;
unsigned short ss0, __ss0h;
unsigned int esp1;
unsigned short ss1, __ss1h;
unsigned int esp2;
unsigned short ss2, __ss2h;
unsigned int cr3;
unsigned int eip;
unsigned int eflags;
unsigned inteax, ecx, edx, ebx;
unsigned int esp, ebp, esi, edi;
unsigned short es, __esh;
unsigned short cs, __csh;
unsigned short ss, __ssh;
unsigned short ds, __dsh;
unsigned short fs, __fsh;
unsigned short gs, __gsh;
unsigned short ldt, __ldth;
unsigned short trace, bitmap;

you are creating 2 of each variable, so your TSS struct is just plain wrong

[apamment]

i set that when doing a task switch

Code: Select all

switch_to:
	mov ecx, [current_task]	
	mov esp, [ecx]

	mov ebx, [ecx+8];put content of the k-stack field into ebx.
	mov [system_tss+4],ebx ;update system tss. (esp)

	mov ebx, [ecx+16] 	
  	mov cr3, ebx

	mov al, 0x20
	out 0x20, al
	pop gs	 
	pop fs
	pop es
	pop ds
	popad

	add esp, 8

	iret

[Combuster]

you are creating 2 of each variable, so your TSS struct is just plain wrong

For each segment field (2 bytes) in the TSS there is unused padding (another 2 bytes). Doesn't look wrong to me.

[BASICFreak]

you are creating 2 of each variable, so your TSS struct is just plain wrong

For each segment field (2 bytes) in the TSS there is unused padding (another 2 bytes). Doesn't look wrong to me.

OK, I'll admit when I'm wrong. I'm used to using uint32_t on these and didn't see these are 16 bit fields in his struct.

@apamment:
I would step the code with Bochs and watch the stack and registers as the system call enters and exits.

[apamment]

I haven't made any progress, but I thought I would post my investigations so far...

upon entering a syscall, the registers are:

eax: 0x7 ebx: 0xffffffff ecx: 0xffffffff edx: 0xffffffff
edi: 0 ESI:0 ebp:f0201fd8 esp: 31001fe4
GS: 31000023 FS: 100023 ES: 100023 DS: 100023
Int no: 30 eflags 296

ebx, ecx, edx, gs, fs, es, ds are all wrong.

ebx should be 0, ecx should be a memory location, edx should be 1

the stack looks ok:
30 (int no)
0 (error code)
400002f4 (eip)
1b (cs)
296 (eflags)
f021fb8 (user stack ptr)
23 (stack segment)

upon exiting the syscall all is the same as entering, except eax which is 0xffffffff (which is correct given that edx previously was 0xffffffff)

once returning to the task it page faults, EIP is 0xffffffff

UPDATE: it seems like the entire user stack is filled with 0xffffffff.. that can't be right?

[apamment]

I fixed it

was a problem in my memory manager where i was allocating the user stack!