INVLPG doesn't work but reloading CR3 does

[lexected]

Hello everybody,

I'm hitting annoying problem related to INVLPG instruction. Here is my code:

Code: Select all

void * cr3_mem = mem_alloc();
void * workaddr = (void*)&kernel_pgsetup;
page_map(workaddr, cr3_mem, PSTRUCTP_STDPG_KERNEL);
asm("invlpg %0" : : "m" (cr3_mem));
pdir_init(workaddr, cr3_mem);

I checked page_map and it maps memory as expected, but INVLPG seems to not have any effect on a relevant part of memory.
Generally, I always allocate a page for new directory, then I map it (always, maybe here the problem is) to &kernel_pgsetup (which is a symbol defined in linker script). When I allocate and map pdir for the first task, mapping works, but when I do it for the second time, the new memory (allocated by mem_alloc() directly after the first) is not changed and I think &kernel_pgsetup is still mapped to the previous. The problem may lay in mapping all new page directories to the same virtual page, but I'm not sure.

When I replace INVLPG with

Code: Select all

asm("movl %%cr3, %%eax\n"
	"movl %%eax, %%cr3" : : : "%eax");

(flushing the whole actual TLB), everything works fine.

Where could the problem be? Is it incorrect to have always one page for editing new page directories and tables? Can I do something else?
(I'm testing everything in bochs+gdb)

Thanks,
Filip.

[iansjack]

Try replacing "(cr3_mem)" with "(*cr3_mem)".

[Brendan]

Hi,

Where could the problem be? Is it incorrect to have always one page for editing new page directories and tables? Can I do something else?

Are you using the "recursive paging trick" (e.g. where a page directory is used as a page table to turn part of the virtual address space into a mapping of all page tables)? If you are; then whenever you modify page directory entries you need to invalidate the effected area in the normal part of the virtual address space (outside the mapping) and you also need to invalidate the effected area within the mapping.


Cheers,

Brendan

[lexected]

Try replacing "(cr3_mem)" with "(*cr3_mem)".

Well, gcc inline assembly is a bit of mistery for me, but cr3_mem seems to be correct.

Are you using the "recursive paging trick" (e.g. where a page directory is used as a page table to turn part of the virtual address space into a mapping of all page tables)? If you are; then whenever you modify page directory entries you need to invalidate the effected area in the normal part of the virtual address space (outside the mapping) and you also need to invalidate the effected area within the mapping.

In this code, I already have existing ptable and I only edit one pte. I tried all possible combinations of INVLPGs (even in a loop), but it didn't help me.

What's interesting is, that everything works in VMWare player. Bochs still fires #PFs. Is it possible to bochs to have a bug in such instruction?

[stlw]

What's interesting is, that everything works in VMWare player. Bochs still fires #PFs. Is it possible to bochs to have a bug in such instruction?

cr3_mem is virtual address or physical ?
if it is physical address (as cr3 location supposed to be) you need to go and re-read INVLPG documentation and understand better what it does and why.

Stanislav

[iansjack]

Well, gcc inline assembly is a bit of mystery for me, but cr3_mem seems to be correct.

It would take but a few seconds to try it and see. But if you have a better way to spend those few seconds ....

[Griwes]

It should be

Code: Select all

asm("invlpg (%0)" : : "m" (cr3_mem));

Looks like the Inline Assembly Examples wiki page got the instruction wrong... Both my kernel and linux use the `invlpg (%0)` version, and so says the Paging wiki page (which refers to the linux kernel...), so I am going to assume that this is the correct way to encode this instruction.

(I think that) the Intel Manual agrees:

The source operand is a memory address.

[iansjack]

Interesting. In my code I use expressions similar to:

Code: Select all

asm ("invlpg %0;"
                :
                :"m"(*(char *)TempUStack)
);

and they work perfectly! But I make no claim that they are right. Perhaps it doesn't matter. I do remember that I had a lot of trouble getting inline INVLPG instructions to work properly so I think I'll leave mine as they are, as they work.

But, in that case, I have no idea why it isn't working for the OP. I still think it's worth trying both forms, but if it's too much trouble ....

[Griwes]

There's a possibility that it's a thing with the inline assembly's "m", but I do not feel strong enough to venture into GCC docs to find out.

[Owen]

Code: Select all

const char *ptr = <address of thing to be invalidated>;
__asm volatile("invlpg %0" :: "m"(*ptr));

Is the correct use of invlpg and constraints. It'll probably expand to something like

Code: Select all

invlpg <address of thing to be invalidated>

or

Code: Select all

movl $<address of thing to be invalidated>, %eax
invlpg (%eax)

The following is wrong:

Code: Select all

const char *ptr = <address of thing to be invalidated>;
__asm volatile("invlpg (%0)" :: "m"(ptr));

because it will break when the m constraint doesn't evaluate to a single register

However, both of them are incorrect because they're missing something important:

Code: Select all

const char *ptr = <address of thing to be invalidated>;
__asm volatile("invlpg %0" :: "m"(*ptr) : "memory");

so the compiler's optimizers might make unwarranted optimizations.

Note that Linux' actual use is

Code: Select all

static inline void __native_flush_tlb_single(unsigned long addr)
{
        asm volatile("invlpg (%0)" ::"r" (addr) : "memory");
}

which is correct on both counts (because it uses the "r" specifier, not the "m" specifier) - although also note that the "m" specifier is slightly preferable for allowing the optimizer more flexibility

[Griwes]

And this, children, is why the compiler should understand the assembly - and why this broken inline assembly syntax is broken...

[lexected]

cr3_mem is virtual address or physical ?
if it is physical address (as cr3 location supposed to be) you need to go and re-read INVLPG documentation and understand better what it does and why.

Sorry for the mistake, it should be virtaddr. When I was writing this post, my code already contained reloading cr3 and I wrote it wrong here.

I would like to focus on why is it working in VMWare, but not in bochs? The same code, the same build, two different disk formats but everything else is same.

Owen, Griwes, iansjack:
What does the ModRM following the INVLPG contain? The location of the address I want to invalidate in TLB, or the address itself? I think it's the first (below).

I still think it's worth trying both forms, but if it's too much trouble ....

My actual combination:
IN: asm("invlpg %0" : : "m" (virtaddr));
OUT: invlpg 0x8(ebp)

Combination 1:
IN: asm("invlpg %0" : : "m" (*virtaddr));
OUT: invlpg 0x8(ebp)
warning: dereferencing ‘void *’ pointer [enabled by default]

Combination 2:
IN: asm("invlpg (%0)" : : "m" (virtaddr));
Error: missing ')'
Error: junk `(%ebp))' after expression

Combination 3:
IN: asm("invlpg (%0)" : : "m" (*virtaddr));
Error: register value used as expression

[stlw]

Owen, Griwes, iansjack:
What does the ModRM following the INVLPG contain? The location of the address I want to invalidate in TLB, or the address itself? I think it's the first (below).

The virtual address you want to invalidate in the TLB

[lexected]

The virtual address you want to invalidate in the TLB

No, that doesn't make sense. It would mean that everytime you want to invalidate some page, you must edit ModRM of that instruction. That's not possible.

[iansjack]

From the Intel manual

Invalidates (flushes) the translation lookaside buffer (TLB) entry specified with the source operand. The source operand is a memory address. The processor determines the page that contains that address and flushes the TLB entry for that page.

So, sensible or not, it is the virtual address that the page maps to. (Well, you can use any virtual address within the page so it would be more accurate to say "a virtual address".)