Remapping pages vs. memcpy

[stack]

Is it feasible to use the paging machinery to remap the memory of a process? The alternative is copying data. 4K granularity is fine.

A few years ago I came across a discussion somewhere about this, in the context of a Linux kernel. If I remember correctly, the consensus was that the overhead of kernel IPCs and possibly the side-effects (caching, TLB, etc) make copying more attractive.

Assuming the client code is tightly integrated with the OS, what do you think? Are there additional issues with moving code around this way (other than the obvious need to fix-up dangling pointers)?

Thank you for your time.

[Octocontrabass]

Is it feasible to use the paging machinery to remap the memory of a process?

There's some point where moving larger amounts of memory will be faster using remapping instead of memcpy. That point moves depending on how well you optimize each of those functions.

Assuming the client code is tightly integrated with the OS, what do you think?

There are optimizations that may not be possible in the general case but significantly reduce overhead when they are possible. For example, you can get rid of most of the TLB flushing overhead when the client or the CPU is single-threaded. A client that's aware of these situations can dynamically choose whether to call memcpy or to call the OS to remap pages.

Are there additional issues with moving code around this way (other than the obvious need to fix-up dangling pointers)?

Code is usually not designed to be moved at runtime. You can move it when you load it, but why not load it at your desired address in the first place?

[nullplan]

Creating a second mapping of the same physical memory has different semantics from having an independent copy. You'd need to implement copy-on-write semantics when doing that, and that means you suddenly get into the realm of TLB shootdowns when implementing this (since you need to revoke the write permission on at least the source page(s)), and that massively tips the scales towards just copying the memory.

Not to mention that it is possible that the application can be optimized not to require a giant memcpy().

[stack]

Code is usually not designed to be moved at runtime. You can move it when you load it, but why not load it at your desired address in the first place?

My project involves code and data which can be garbage collected. Relocation and fixup require some finesse, but are not too much of an issue. I am investigating the possibility of using remapping to reclaim memory, something which would seem to require cooperation from the OS.

[Ringding]

The Azul JVM does/did this.

[stevefan1999]

From https://en.wikipedia.org/wiki/Accent_kernel:

In 1979 one of the Aleph engineers, Richard Rashid, left for CMU and started work on a new version of Aleph that avoided its problems. In particular, Accent targeted workstation machines featuring a MMU, using the MMU to "copy" large blocks of memory via mapping, making the memory appear to be in two different places. Only data that was changed by one program or another would have to be physically copied, using the copy-on-write algorithm.

So some kind of memory protection mechanism must be involved here.

I'm experimenting on persistent data structure to replace MMU in some degree by structural sharing, but it makes the data very hard to debug. Plus, it seems like with MMU I still can't do mmap for persistent data structures where its pointers/leaves stayed on so I still have to rethink about it in some way.

[AndrewAPrice]

I'm doing this for my RPC.

My RPC has 'messages' and 'mini-messages'. Messages are arbitary sized and aligned to physical pages, and sending them to another process 'gifts' them the page. Mini-messages are fixed size messages that can fit in 4 64-bit registers (32-bytes or less).

A string to the VFS saying to open this file on this path is a arbitary sized message. A 2d vector of which direction the mouse moved is a mini-message.

Arbitrary sized messages can be as large as they want, but the most common size will be 4KiB, so I want to focus on optimizing these single page messages. One optimization: in each user process when we 'free' pages, I keep a pool of recently freed pages without actually releasing them to the OS.