making 256TB user process vmem ,,,, is this possible??

[criny]

I'm know devloping linux-like new model kernel just for fun...

I want to make user process use full virtual memory space 256TB in x86_64 machine...

I can't get answer now...

also... I'm not newbe in making kernel...

IDTR must indicate virtual memory address to execute interrupt handler......... and.... many thing there is to make system call environment..I know....

hm..... does anyone know about this????

may be my question look so fool.... but I want to make it possible if x86 mahchine supports it.....


Give me some Good idea guys~~~ XD

[bluemoon]

Not so simple.

First, let's look at how much address space on x86_64: it's 256T, but then you would reserve some for kernel and page stuff, so technically, no, you only have 256T - "kernel space", although you may squeeze the kernel to tiny bits.

But don't give up, the above scenario assume the question was giving 256T space access which can be accessed directly. There are other technique like overlay/swap to provide even more accessible memory to an application, if you really need that much.

[criny]

thanku guy....
i already have enough knowledge in x86 architecture... so I know it may be difficult.... because I haven't saw the kernel supporting fully-256TB-virtual-memory-user-process ... even in educational kernel program...

I want to make it possible just because of making kernel model sooooooooooooo easy.....XD

[criny]

I made it possible just ago XD

[SpyderTL]

64-bit Long mode requires paging to be enabled, so you would have to define 262,144 pages (4MB pages) for each application, including the kernel.

That's over 4MB of page tables for each application.

Am I right about this?

[stlw]

64-bit Long mode requires paging to be enabled, so you would have to define 262,144 pages (4MB pages) for each application, including the kernel.

That's over 4MB of page tables for each application.

Am I right about this?

There are no 4MB pages in long mode.
But servers with sooo much memory usually don't stop on available 2MB and use 1Gb pages.

Stanislav

[SpyderTL]

You are correct. I meant 4KB pages, which is even worse. Although the page entries are only 8 bytes, not 16 as I was using above.

So that would be... 549 GB of page tables for each application...

So, how do you get 1 GB pages?

[Combuster]

64-bit mode supports at least 4K and 2M pages. 1G pages is a more recent feature. You'd set it just like the size bit for 2M pages but then on a higher table.

There might be some CR4/EFER things involved but I haven't checked.

[SpyderTL]

Found this interesting article about performance comparisons between 4KB, 2MB and 1GB page sizes, if anyone is interested.

http://www.pvk.ca/Blog/2014/02/18/how-b ... -pages-be/

But back to my original question, if you were going to give applications a 256 TB address space, does that mean that each application would need over 500 GB of page tables? If so, I guess that makes 4KB page tables impossible.

We need a list of all of the different page directory formats and their maximum address space. Is that already in the wiki somewhere?

Just running the numbers, it looks like:

4KB = 4GB
4KB + PAE = 4GB?
PSE (4MB) = 4TB?
PSE + PAE (2MB) = 2TB?
1GB = 1PB?

While larger page table sizes allow you to define "higher" page addresses, you are still limited to 1,048,576 total page entries, regardless of which page table format you use.

Is this correct?

I found all of the table format information on Wikipedia: http://en.wikipedia.org/wiki/Page_(computer_memory)

According to that page, these are the only page table formats available on x86 and x86-64 platforms. Itanium has several additional formats, but I'm ignoring those.

Am I missing something, or is the only way to provide an application with a real 256TB address space to use 1 GB pages (on very new hardware)?

[dschatz]

Everyone seems to be missing the fact that you can fault in the address space, so you can just unmap regions and reuse the page tables to map in new regions. So I don't think supporting a large address space is a problem from this perspective.

[SpyderTL]

Does that mean that, technically, you could just have 1 4KB page, and you could just swap in the requested page and at the same time "swap" in the requested page table entry and then jump back to the faulting instruction?

I know it would be slow, but technically would it work?

[iansjack]

No. At the very least you need an unchangeable page for the fault handler.

[Brendan]

Hi,

Does that mean that, technically, you could just have 1 4KB page, and you could just swap in the requested page and at the same time "swap" in the requested page table entry and then jump back to the faulting instruction?

I know it would be slow, but technically would it work?

In the virtual address space; you must have:

• An IDT with a minimum of 14 entries
• A TSS (at least the stack related parts of a TSS)
• A GDT with a minimum of 4 entries (excluding the "NULL" entry which can overlap something else)
• Interrupt handlers (that may be minimal stub/s that do little more than switch to the kernel's virtual address space)
• A (potentially tiny) stack that the interrupt handler/s can use

In theory (with severe performance implications) all of that can be crammed into a single 4 KiB page. Note: If you run applications at CPL=0 then you can do it without a TSS, but all the rest is still needed and it'd still need a minimum of a single 4 KiB page.

In addition to that; consider an instruction like "push dword [0x1234FFFF]". For this instruction to succeed (without page faults); at a minimum the code at RIP, the top of the stack, and the data at address 0x1234FFFF must all be in the virtual address space at the same time. There are other "3 memory location" instructions (e.g. "movsd") but there are no "4 memory location" instructions (that I can think of). However, all of the locations maybe be misaligned; such that the CPU needs 2 pages to fetch the entire instruction, 2 pages to access each piece of data, etc.

This means that a minimum of 7 pages (2 for code, 2 for data, 2 for user stack or data, plus one for kernel) need to be mapped into the virtual address space to guarantee that all possible instructions can be executed successfully and all exceptions can work correctly.

I want to make user process use full virtual memory space 256TB in x86_64 machine...

An alternative is to have "thread specific storage". For example, if you split the virtual address space into 1 GiB of "process space", 2 GiB of "thread space" and 1 GiB of "kernel space"; then a process with 131072 threads would end up having a total of "1 + 2*131072 GiB" of virtual memory (or slightly more than 256 TiB of virtual memory) and you wouldn't even need a 64-bit CPU to do that.


Cheers,

Brendan