OSDev.org

I've always set out my per-CPU data with four consecutive pages:

• GDT, TSS, other per-CPU data (e.g. pointer to current thread), CPU emergency stack space (For double faults & NMIs - NMIs including panic NMI IPIs)
• IOPB for ports 0 - 32767
• IOPB for ports 32768 - 65536
• End of IOPB guard page. 3 bits following IOPB must be all ones; the all-bits-set shared page is mapped here as it must already exist.

The all-bits-set shared page is mapped into the IOPB slots for processes which have no IO permission bitmap of their own, denying all privileges.

OK, it seems like a smart idea to have the IO permission bitmap per process instead of per thread. I could easily make the V86 thread a new process instead of just an ordinary kernel-thread.

I have a somewhat different layout of the per-CPU data today:

page 1 + most of page 2: per-core data (requires this much space because of timer data structures)
last 0x48 bytes of page 2: per-core GDT entries (especially the selector that is mapped to the per-core data)
page 3-17: shared part of GDT

I could change it to something like this:

page 1: core private stack + double-fault TSS + core TSS
page 2: IO permission bitmap for core TSS, part 1
page 3: IO permission bitmap for core TSS, part 2
page 4: a number of 1s + first part of per-core data
page 5: second part of per-core data + 0x70 bytes of private GDT (which includes the core private selector, a new core stack selector, a new core TSS selector, a new writable core TSS alias selector, a new double-fault TSS and alias selector)
page 6-20: shared part of GDT

When changing process, the scheduler would need to change mappings of page 2 and 3. On every thread change, it would need to update the ESS0 field in the core TSS.

An alternative approach would be to pre-allocate two pages with the default IO permission bitmap, and keep those two physical addresses in the thread control block (IOW,
it would still be per-thread). When a new thread is created, it would simply inherit the two pages. When the V86 bios thread wants its own (allow all) bitmap, it simply
allocates a 8k memory block, fills it with 0s, and places the two pages in the thread control-block. The scheduler would then remap page 2 and 3 if any of the physical
addresses changes instead, and if they do, force a CR3 flush.

This would only consume two GDT entries for TSS regardless of number of CPUs in the system.

Adding the double-fault TSS to the private core data makes it possible to have one double-fault TSS per core, which eliminates the problem of several cores hitting double-fault at the same time triggering tripple-faults to to busy double fault TSS. The IDT entry for double fault would be a task gate pointing to a fixed TSS selector, which has different mappings in different cores.

Alternative solution:

Instead of checking if a TLB-flush is needed (takes time), an alternative approach is to always do a TLB-flush, and add some additional benefits to the design (eliminating the per-core TSS and the extra selector per thread for the kernel stack):

page 1: aliased thread kernel stack
page 2: aliased thread TSS
page 3: IO permission bitmap, part 1
page 4: IO permission bitmap, part 2
page 5: a number of 1s + core private stack + double-fault TSS
page 6: first part of per-core data
page 7: second part of per-core data + 0x78 bytes of private GDT (which includes the core private selector, new aliased thread kernel stack, a new core stack selector, a new aliased core TSS selector, a new selector for modifying core page-tables, a new double-fault TSS and alias selector)
page 8-22: shared part of GDT

This design does not need to modify ESS0 in the TSS (the real thing is aliased), and it doesn't need to beware of CR3 and LDT in TSS handlers (double-fault). The scheduler would replace 4 page tables (page 1-4) from the thread-control block and then always do a CR3 flush. Threads would only need one GDT entry (the thread control-block). The drawback is that the kernel-debugger (and similar) would need special logic to decode contents on the kernel stack. It will have the same GDT selector in all threads, and in order to read it there is a need to map the physical address in the thread control-block to a linear address, and then read the contents.