Page 1 of 2

IDT in assembly.

Posted: Thu Nov 03, 2011 10:37 am
by sandras
Hello, World!

As this is my first post, let me briefly introduce myself. I have been fascinated by and interested in computers ever since I was a kid. More recently I have been looking into operating systems. Most of them are ugly, but some I find particularly beautiful. Eventually I started looking into creating one of my own. I am still in research and design mode mostly. Anyway, let me explain my problem. While I was trying to setup IDT in assembly I had this code.

Code: Select all

extern	isr1
extern	isr2
...
extern	isr47

lidt	[idtd]

%macro IDTENTRY 1
	dw		isr%1 & 0xffff
	dw		0x08	
	db		0
	db		10001110b
	dw		isr%1 >> 16
%endmacro


idtd:
	dw		375
	dd		idt


idt:
	IDTENTRY	1
	IDTENTRY	2
	...
	IDTENTRY	47
While compiling with NASM it gave me this error.
entry.a:132: error: `&' operator may only be applied to scalar values
entry.a:132: error: shift operator may only be applied to scalar values
...
I searched on the web and found this. http://wiki.osdev.org/I_Cant_Get_Interr ... 2_mean_.3F

So I modified the code to this.

Code: Select all

%define SECTIONBASE 0x100000

extern	isr1
extern	isr2
...
extern	isr47

lidt	[idtd]

%macro IDTENTRY 1
	dw		(SECTIONBASE isr%1 - $$) & 0xffff
	dw		0x08	
	db		0
	db		10001110b
	dw		(SECTIONBASE isr%1 - $$) >> 16
%endmacro


idtd:
	dw		375
	dd		idt


idt:
	IDTENTRY	1
	IDTENTRY	2
	...
	IDTENTRY	47
Now I get this error.
entry.a:132: error: expecting `)'
entry.a:132: error: expecting `)'
...
Does anyone know how to fix it? Sorry, if it's a dumb question, but I'm out of ideas. And perhaps the Wiki should be updated?

Have a nice day. : )

Re: IDT in assembly.

Posted: Thu Nov 03, 2011 10:45 am
by Bietje
Sandras wrote: I searched on the web and found this. http://wiki.osdev.org/I_Cant_Get_Interr ... 2_mean_.3F

So I modified the code to this.

Code: Select all

%define SECTIONBASE 0x100000

extern	isr1
extern	isr2
...
extern	isr47

lidt	[idtd]

%macro IDTENTRY 1
	dw		(SECTIONBASE isr%1 & 0xffff  ; where the hell is the closing bracket?
	dw		0x08	
	db		0
	db		10001110b
	dw		(SECTIONBASE isr%1  - $$) >> 16
%endmacro


idtd:
	dw		375
	dd		idt


idt:
	IDTENTRY	1
	IDTENTRY	2
	...
	IDTENTRY	47
Now I get this error.
entry.a:132: error: expecting `)'
entry.a:132: error: expecting `)'
...
Does anyone know how to fix it? Sorry, if it's a dumb question, but I'm out of ideas. And perhaps the Wiki should be updated?

Have a nice day. : )
That is, because you are missing a ')'.. See comment in your code above.

Re: IDT in assembly.

Posted: Thu Nov 03, 2011 10:52 am
by sandras
I'm sorry, the bracket went missing accidentally, when I pasted the code and fiddled with it. The bracket is actually there in the source code. And the error is still there. I Edited the original post to include the bracket.

Re: IDT in assembly.

Posted: Thu Nov 03, 2011 11:36 am
by Brendan
Hi,
Sandras wrote:Does anyone know how to fix it?
There was a bug in the wiki's example code, which should've been obvious.

The idea is to subtract the address of the start of the section (e.g. "foo - $$"), then compensate by adding a scalar value that is equal to the address of the start of the section (e.g. "(foo - $$) + SECTIONBASE").

Now look at your "(SECTIONBASE isr%1 - $$)" and see if you can spot the missing operator.


Cheers,

Brendan

Re: IDT in assembly.

Posted: Thu Nov 03, 2011 11:54 am
by sandras
That's what I tried before posting.

Code: Select all

dw		((isr%1 - $$) + SECTIONBASE) & 0xffff
dw		0x08
db		0
db		10001110b
dw		((isr%1 - $$) + SECTIONBASE) >> 16
But it still gave me an error.
entry.a:132: error: `&' operator may only be applied to scalar values
entry.a:132: error: shift operator may only be applied to scalar values
...
Am I not getting something?

Re: IDT in assembly.

Posted: Thu Nov 03, 2011 12:42 pm
by turdus
May I suggest to use a different assembler? This and other problems make nasm annoying, not to mention the ughly mess of it's codebase.

I suggest to fasm: http://www.flatassembler.net/

Benefits:
1. much more complex macro capabilities (it can handle what you're going to do, concatenate a string and a macro argument and use the result as a label)
2. macro syntax is more convenient
3. it's much faster and smaller
4. supports more instruction
5. fasm can compile itself, which makes porting to your OS easy

If you cannot change assembler, that's a pity, hopefully somebody can help you to find a workaround.

Re: IDT in assembly.

Posted: Thu Nov 03, 2011 1:58 pm
by egos
If isr* absolute address (as scalar value) is imported then its correction is not required. If isr* offset in section (as scalar value) is imported you should add to it the section base.

If this not work, try to export/import low and high word of isr* address separately.

turdus, fasm rules!

Code: Select all

virtual
desc KCODE,irq0_dummy,DF_INT32
load IRQ0_GATE qword from $-8
end virtual

Re: IDT in assembly.

Posted: Fri Nov 04, 2011 4:48 am
by Combuster
The thing you are trying to do only works when your kernel is a single-file all-assembly flat binary. Any intermediate representation will have to emit information that the value at this and that location will have to be replaced by some other value. In your case you can't provide a value, but instead you need an equation, which is something object files can not do, and switching to FASM can not possibly fix that (even though its users believe so because they have that habit of limiting themselves to that exact niche where it can be pulled off).

The only portable solution is to write a function that fills out the IDT at runtime.

Re: IDT in assembly.

Posted: Fri Nov 04, 2011 9:50 am
by sandras
Hey, I tried putting my ISR handlers in the same file as my IDT table and now it seems to work - the file assembles and lidt instruction seems to register the IDT table. I think I understand the problem I had a little more. The next problem now is that I get exception 9. I tried finding out what causes the exception and I think the interrupt 9 is fired when my hang loop starts. I don't know why in the world anything in that loop would cause the exception.

Code: Select all

hang:
	call prntdebug
	hlt
	jmp hang
My console output.
debug
received interrupt: 9
error code: 0
debug
I tried removing the hlt instruction and having just a simple loop without nothing in it and the interrupt still fires, so it is not the hlt instruction that is causing it. I suppose the loop itself couldn't cause the interrupt 9, could it?

I also found this http://f.osdev.org/viewtopic.php?f=1&t=11002&start=0. So I thought that maybe something is wrong with my keyboard handler, but I do not press any buttons before the exception 9, so the interrupt 33 can't be fired, so it can't be the keyboard handlers fault, can it?

Also, the strange thing is not only that the interrupt 9 fires, but that my PIT handler does not get executed and the same goes for my keyboard handler - the keyboard does not work. Or maybe my PIT handler does get executed and that is what causes the exception? But why the exception? I can't find anything that would cause it. Besides, the handler code is the same as when I had my IDT setup in C and it worked fine then. And could the exception 9 disable interrupts, so that neither the PIT nor the keyboard handlers could be executed? I don't think so (Actually, after I wrote almost all of this, I inserted an int instruction after the hlt one in the hang loop and the interrupt does occur after the exception 9 (This raises another question for me - how does the hlt instruction work? Why does the int fire an interrupt if the hlt is supposed to halt the computer and, as I understand it, do nothing?)). I double checked and the interrupt handler does re-enable the interrupts. (Also, on a side note, I was wondering - doesn't the processor itself disable interrupts when an interrupt is fired and re-enable them when you return from one using iret instruction? I could swear I read that it does somewhere. (Actually, again, I tried out the handler without sti at the end and the processor seems to automatically enable interrupts when the interrupt handler returns. So is JamesM's tutorial wrong on this, as I partially based my kernel on it? I heard the are bugs.)) Here's my assembly (With all the cli and sti not yet removed) as well as C parts of the interrupt handler.

Code: Select all

commonisr:
	pusha ; Pushes edi, esi, ebp, esp, ebx, edx, ecx, eax.
	mov ax, ds ; Lower 16-bits of eax = ds.
	push eax ; Save the data segment descriptor.
	mov ax, 0x10 ; Load the kernel data segment descriptor.
	mov ds, ax
	mov es, ax
	mov fs, ax
	mov gs, ax
	call handlisr
	pop eax ; Reload the original data segment descriptor.
	mov ds, ax
	mov es, ax
	mov fs, ax
	mov gs, ax
	popa ; Pops edi, esi, ebp, ...
	add esp, 8 ; Cleans up the pushed error code and pushed ISR number.
	sti
	iret

%macro ISRNOERRCODE 1 ; Define a macro, taking one parameter.
global isr%1 ; %1 accesses the first parameter.
isr%1:
	cli
	push	byte 0
	push	byte %1
	jmp commonisr
%endmacro

%macro ISRERRCODE 1
global isr%1
isr%1:
	cli
	push	byte %1
	jmp commonisr
%endmacro

ISRNOERRCODE	 0
ISRNOERRCODE	 1
...
ISRNOERRCODE	47

Code: Select all

void handlisr(registers_t regs)
{
	int	scan;

	if(regs.intrno < 32)
	{
		prntstr("received interrupt: "	);
		prntstr(int2str(regs.intrno));
		prntc('\n');
		prntstr("error code: ");
		prntstr(int2str(regs.errcode));
		prntc('\n');
		if(intrhandl[regs.intrno] != 0)
	}else{
		if(regs.intrno == 32)
		{
			prntstr("tick: ");
			prntstr(int2str(tick));
			prntc('\n');
			tick++;
		}
		if(regs.intrno == 33)
		{
			scan = ib(0x60);
			if(scan & 128)
			{
				// Released.
				// Codes -86 and -74 stand for reeased left and right shift keys respectively.
				if(scan == -86 || scan == -74)
				{
					shift = shift-1;
				}
			}else{
				// Pressed.
				// Codes  42 and  54 stand for pressed left and right shift key respectively.
				if(scan == 42 || scan == 54)
				{
					shift = shift+1;
				}else{
					if(shift == 0)
					{
						prntc(lowercase[keyno]);
					}else{
						prntc(uppercase[keyno]); 
					}
				}
			}
		}
		// Send an EOI (end of interrupt) signal to the PICs.
		// If this interrupt involved the slave.
		if(regs.intrno >= 40)
		{
			ob(0xA0, 0x20);						// Send reset signal to slave.
		}
		ob(0x20, 0x20);							// Send reset signal to master.
	}
}
This might sound silly, but does ALL the kernel have to be in a single assembly source file and does it REALLY have to be a flat binary file to work? Because the object files link fine and the exception handler works without a problem too.

Also I had a look again at the IDT layout. Again, mine is like this.

Code: Select all

%macro IDTENTRY 1
	dw		(SECTIONBASE + isr%1 - $$) & 0xffff ; Base low.
	dw		0x08	 ; Selector.
	db		00000000b ; Always zero.
	db		10001110b ; Flags.
	dw		(SECTIONBASE + isr%1 - $$) >> 16 ; Base high.
%endmacro

idt:
	IDTENTRY	 1
	IDTENTRY	 2
	...
	IDTENTRY	47

And it seems fine. I know I'm practically asking you to check it, but I already checked it several times and if there is something wrong, I keep missing it. Am I possibly setting any values so that my ISR handlers would be called incorrectly?

And one last thing. I did not want to do it the equation way, that Combuster suggested, as it seems inelegant IMO and I think it would make the kernel bigger and yes, I know that it's just a couple of bytes, besides, it is boot code, which is executed only once, thus not worth the effort trimming in most peoples opinion, but I'm a little OCD about it and I think it all adds up and if you don't save bytes from the beginning, you will have a noticeably bigger kernel. This goes with no disrespect to you, Combuster, as what you said helped me and I have read these forums for quite a while and think you are one of the more knowledgeable people around here.

Hope there's not too much questions and code. : )

Re: IDT in assembly.

Posted: Fri Nov 04, 2011 10:20 am
by egos
Combuster, you are right. Modern object file formats have no support for 16-bit links or scalars (constants). Scalar values can be useful only in same module where they are defined.

Code: Select all

label irq0_dummy at ($-$$)+FIXED_BASE_OF_UNIQUE_SECTION
iret

virtual
desc KCODE,irq0_dummy,DF_INT32
load IRQ0_GATE qword from $-8
end virtual

Re: IDT in assembly.

Posted: Sun Nov 06, 2011 9:27 pm
by sandras
OK, I've got it.

First off all, I had my IDT table starting from IDTENTRY 1, not IDTENTRY 0. When CPU receives an interrupt 0, it looks for the IDT entry number 0 in the IDT table. And in my case IDT entry number 0, was actually pointing to the ISR 1, not 0. Thus when I had exception 9 occuring, it actually was exception 8. A stupid mistake.

Then I had exception 13 with error code 580 occur in an infinite loop.

Anyway, finally, I decided to check the IDT implementation in C that I had. There it was - IRQ remapping. I remapped the IRQs in assembly. And now it works.

Oh, and there were some memory setting in the C code too, which I didn't translate to assembly yet, as it works without it and I don't understand what it is for. I mean I do understand that it is for setting the memory to, in this case, 0 of the IDT entry's and interrupt handler function, but should one do that? I mean, why should one clear the IDT entry's and interrupt handler function if those resources will be needed? Besides, the code worked with all the clearing which doesn't make sense to me. Can anyone briefly explain this to me? I'm clearly misunderstanding something here. Neither the tutorials nor this forum provide any information on that. Internet, including osdev wiki, is of no use too in this case.

Here's the idt.h.

Code: Select all

// A struct describing an interrupt gate.
struct idtentrystruct
{
	short addrlo; // The lower 16 bits of the address to jump to when this interrupt fires.
	short sel; // Kernel segment selector.
	char always0; // This must always be zero.
	char flags; // More flags. See documentation.
	short addrhi; // The higher 16 bits of the address to jump to.
} __attribute__((packed));


// A struct describing a pointer to an array of interrupt handlers.
// This is in a format suitable for giving to 'lidt'.
struct idtptrstruct
{
	short size;
	int addr; // The address of the first element in our idtentry_t array.
} __attribute__((packed));


typedef struct idtentrystruct	idtentry_t;
typedef struct idtptrstruct	idtptr_t;
And the idt.c.

Code: Select all

idtentry_t	idtentries[256];
idtptr_t	idtptr;

extern		isr_t intrhandl[];					// Extern the ISR handler array so we can nullify them on startup.


void idtgate(int num, int base, int sel, char flags)
{
	idtentries[num].addrlo = base & 0xFFFF;
	idtentries[num].addrhi = (base >> 16) & 0xFFFF;
	idtentries[num].sel = sel;
	idtentries[num].always0 = 0;
	idtentries[num].flags = flags;
}


void initidt()
{
	idtptr.size = sizeof(idtentry_t) * 256 -1;
	idtptr.addr = (int)&idtentries;

	mmset(&idtentries, 0, sizeof(idtentry_t)*256);

	// Remap the irq table.
	ob(0x20, 0x11);
	ob(0xA0, 0x11);
	ob(0x21, 0x20);
	ob(0xA1, 0x28);
	ob(0x21, 0x04);
	ob(0xA1, 0x02);
	ob(0x21, 0x01);
	ob(0xA1, 0x01);
	ob(0x21, 0x0 );
	ob(0xA1, 0x0 );

	idtgate( 0, (int)isr0 , 0x8, 0x8E);
	idtgate( 1, (int)isr1 , 0x8, 0x8E);
	...
	idtgate(47, (int)isr47, 0x8, 0x8E);

	loadidt((int)&idtptr);

	mmset(&intrhandl, 0, sizeof(isr_t)*256); // Nullify all the interrupt handlers.
}
I've gotta say, after all this, I understand the IDT a little bit more, which was one of my goals in the first place, when I decided to rewrite the part of the kernel in question in assembly.

Now, usually, when I expect a reduction in compiled code (I don't like calling it "binary", as everything is binary) size, I compare the size before and after. The result, I have to say, surprised me. The kernel lost 1664 bytes of it's size, which seems quite a lot, when you consider that the functionality is still there, only written in a different language and the code converted was not so big.

And to end the post - thanks for everyone who tried to help me. : )

Re: IDT in assembly.

Posted: Sun Nov 06, 2011 9:57 pm
by gerryg400
The kernel lost 1664 bytes of it's size,
Did you check which section it was 'lost' from. It may be debug code or something else that's not loadable.

With regards to the memset(??, 0, ??), it's not needed because (if the kernel is properly loaded) global variables will already be zeroed.

Code: Select all

 idtgate( 0, (int)isr0 , 0x8, 0x8E);
  
That cast looks ugly. Why are you casting (what I presume is) a function pointer to an int ?

Re: IDT in assembly.

Posted: Sun Nov 06, 2011 10:44 pm
by sandras
My build script (I know I should use make, but I do not NEED to yet, so I don't) automatically strips the kernel.

Here's the output of size oldkernel && newkernel just to show what was lost.
text data bss dec hex filename
4158 180 19526 23864 5d38 oldkernel
text data bss dec hex filename
2482 180 17446 20108 4e8c newkernel
The C IDT code was copied from JamesM's tutorial, so he should be the one who knows why the cast is there. As I said, one of the reasons why I rewrote my IDT code was to better understand the IDT and how to implement the software to set it up. Again, I found that assembly code was more understandable to me, than the C code. I don't say, that the C code provided is good or bad. What I can tell, is that it worked for me. But, in the end, I suppose the cast is there, because idtgate() takes int as the second argument.

Re: IDT in assembly.

Posted: Sun Nov 06, 2011 11:35 pm
by gerryg400
It's better to pass a pointer to the function and cast internally to a uintptr_t to do the bit manipulation. That way you get type checking at the interface. And the cast is hidden in the implementation. The added bonus is that you can have an arch specific idtgate. Your current version will not work in long mode.

I'm concerned about the loss of 2060 bytes of BSS. If you can't explain that then I'm thinking that you've forgotten to define a 2kB data structure (like the idtentry_t idtentries[256];) when converting to asm.

Re: IDT in assembly.

Posted: Mon Nov 07, 2011 12:36 am
by sandras
Is it necessary to have IDT 256 entries long? Will something break if I don't have it this way? And if so, what? I can Imagine that if interrupt fired, and the processor didn't find an entry in the IDT corresponding to that interrupt number and thus could not execute the the ISR, the processor would triple fault. But I assumed the only interrupts that could fire now, when I don't have system calls, are exceptions and some IRQs, the ISR range from 0 to 47.

Checked the wiki and found this.
It can contain more or less than 256 entries. More entries are ignored. When an interrupt or exception is invoked whose entry is not present, a GPF is raised that tells the number of the missing IDT entry, and even whether it was hardware or software interrupt. There should therefore be at least enough entries so a GPF can be caught.
Even though this sort of confirms my thoughts, I would still like to hear your opinion on the subject. Could interrupts above 47 fire?