Thoughts on IST vs no-redzone

[yasar11732]

Hi,

I was working my first osdev project for last one month or so. It was supposed to be 32 bits kernel. I did some initial work such as, setting up GDT, IDT, enabling paging, printf etc. Now I want to start an 64 bit kernel osdev project before I invest too much into 32 bit kernel because it is becoming an obsolete technology (or should I say it is an obsolete technology). Anyways...

I was reading about 64 bit kernels and how it is different than 32 bit ones, I came across this red-zone problem in System V 64 bit ABI. As far as I can see, people recommend -mno-red-zone flag while compiling kernels both in forums and in wiki.

But, it looks like you can specify alternative stack when using 64 bit version of IDT entries. You do this by filling IST bits of IDT entries with number 1 to 7. This number used as an index into Interrupt Stack Table (IST) which is located in 64 bit version of TSS.

I didn't actually try this yet, but it seems like a possible solution for interrupts trashing red-zone. Is there a reason why this is not often mentioned in forums or wiki? Was this overlooked, or simply recommended against. If so, what are the potential drawbacks of using IST with red-zone enabled.

Best Regards,

[Octocontrabass]

Interrupts that share an IST entry can't nest. The later interrupt will overwrite the earlier interrupt's stack. You also only get seven IST entries, and you probably want to reserve some of those for things like NMI and double faults.

If those aren't a problem for your OS, then go ahead and use the IST.

[nullplan]

Yup, as Octo said, the problem is that IST interrupts don't stack. The stack pointer is just hard reloaded with the value from the TSS, and if it already was in use then whatever was there before is now gone. PowerPC has a similar problem, but at least in PowerPC you can recognize the situation and panic accordingly. Here, you cannot recognize that this has taken place, and you will end up in an infinite loop

In practice, though, the absence of a red-zone in kernel is of little concern. Some functions may need a sub/add pair more than if they could utilize it. In a cost-benefit analysis, making it entirely impossible to stack faults and interrupts is probably not worth the instruction savings. There's libgcc, of course, but in practice I just don't link it in, because I don't want to faff about with multilib, and so far have not gotten a linker error. I figure, if I ever do something that requires a libgcc call, I can research the function called and write a quick version in assembler. I don't need it to be fast, only correct.