OSDev.org

I've noticed lately that some of the newer research ukernels out there do not support virtual memory (the storing pages on disk variety, not simply using the MMU for protection and relocation purposes) -- in particular, Singularity and Minix 3.

In the case of Singularity, it sounds like it would be a difficult feature to add given the constraints of their design. For Minix 3, Tanenbaum is arguing that we have enough RAM nowadays that we don't need to spill to disk anymore, but I wonder if his comment is just a cop-out to avoid the problem altogether. From my own attempts I've learned that designing this feature into a ukernel-based OS can be a real pain.

What do people think? Can we kick the whole pagefile/swapfile thing to the curb, or will the apps continue to need more RAM than we have?

I'm just about to go to the office to tend for an application that uses several gigs of swap space. (As in, more than 10 - dunno how much exactly.) Even if you max out a 32bit machine with 4 gigs of RAM, that's easily used up when you're doing Photoshop or something like that. You'd end up limiting the amount of apps the user can open simultaneously, or forcing each app to implement some swapping of their own... I think virtual mem is here to stay.

Of course there are always certain fields where you can ommit it - you won't need swap space in a game console, iPod or the like, where the app developer knows the constraints of the target machine.

In both of my machines at home I have more than 700MB of RAM, and during 'normal' operation I don't need swap space at all. But as soon as I start up several qemu vms (or battlefield2 ), swap space grows to about 2GB, so for me, I'd like to at least have the _option_ to enable swapping.

Furthermore, watching the memory consumption of current applications and especially games, IMO you see that memory consumption grows at the same rate as the average amount of memory installed in computers (today, it's around 1 GB, AFAICS). Just like Solar already said, as soon as there are multiple applications running simultaneously, you'll get short on memory.

But of course I'm not a visionary....

cheers Joe

I like having swap space at hands too. Not as if I don't have enough memory installed in my machine (well, it's 512 mb ecc buffered & registered ram atm - something else my board doesn't take) but it's good to know that I can open more apps at the same time. The OS will swap out stuff if running short of physical memory, as simple as that.

@Colonel Kernel: Well, I 'd implement the swap mechanism as a kind of task in Kernel space which is able to slip into address spaces at needs whilst crawling the list of mapped pages - and I'd have it write pages throu a dedicated swap space driver which in turns talks to the hard disk driver. The decision, which pages are to be swapped out: LRU, ain't it? Further I'd only swap, if we 're going narrow on physical memory.

Hm. I have yet to decide at which level of priority the swapper has to run. Highest? Or beneath everything else? We actually need someone to do dirty work quickly in case we have an urgent need of memory!

Do we really have such generous amounts of RAM available?

32bit machines are limited to 4GB of physical RAM - but usually, motherboard design sets much lower limits. Laptops, for example, are commonly limited to 2GB.

For a test, I booted my Gentoo Linux into an average KDE desktop for developer purposes: Konsole, Kate, Eclipse, Kontact, KWord, Mozilla. 280 MByte were taken afterwards. That is without actually loading any documents or doing something.

Yes, having 1 GB instead of 288 MB would result in zero VM usage. For me. My wife has a 1024 MByte laptop, and her primary app is Photoshop. The thing swaps like mad. Our development server in the company has 50 GB of swap, and runs out of it 1-2 times a month.

Is VM becoming obsolete? No way.

My thoughts:

I'm not sure if general purpose virtual memory is needed anymore. We certainly typically have enough memory these days that normal workstation hardly need to do much swapping.

Do we still want demand loading? If we got demand loading, then extending it to full virtual memory basicly requires having some swap-out policy. Shared memory (if desired) is easier without trying to swap it to disk, though.

For memory hungry applications there's basicly two options. One is the traditional system, the other is to let/require them do self paging which can actually be a good idea, allowing the applications (and servers like database systems) to use their knowledge of the data to control what to swap and when. In many cases this would supposedly lead to better performance than a system wide policy. Object-oriented languages also supposedly tend to spread locality, which reduces paging effectiveness. An object-aware swapper (for example with a compacting garbage collector) could potentially perform much better.

The traditional system has an advantage though: when the system has a few processes that can essentially use as much memory as allowed (say, a database might want to cache as much as possible) a global paging policy can free memory from inactive processes for the active processes. While this might not be necessary, it can sometimes boost performance. Often one has tons of code that's not really going to run at all, but has been brought to (virtual) memory in case somebody actually requests the service they implement.

Also, especially in monoliths, if kernel ever runs out of memory that it needs for it's own purposes, it can just run the pager, and wait for that to free some; this allows more optimistic memory allocation as you don't really need to play safe.

But this is an interesting idea, 'cos in a microkernel demand-loading isn't terribly hard to support, swap-out is much harder. On the other hand, application self-paging is pretty easy to support too, if you're not interested in forcing the application to free pages.

But it all depends on a lot on what kinds of memory policies you want; if one is happy to give free memory to whoever first requests it, and then just report "sorry, out of memory" to the rest of the processes until somebody frees something (or gets killed) then I don't see one losing anything by forgetting about system-level virtual memory, and letting the hungry applications do self-paging.

Reality check: Self-paging sounds nice, but in 99% of all cases what the management worries about isn't performance but how cheap can it be implemented / maintained... and that's where VM (being transparent to the app) scores big time. Not even talking about the requirement to rewrite existing applications... but then again, if we'd worry about that, we wouldn't be doing our own OS projects.

I voted no, the paging mechanism will be always useful.

This sort of thing depends on stuff like if you want your OS to do everything at the same time, then yeah swap is required. My plans with my OS is to only support a small group of task, where a system would be dedicated for this group. As I'm not trying to dominate every OS on the face of the earth, but only accomplishing a specific software field most efficiently and will not support swapping but paging for users applications that requires a very dynamically composed enviroment.

The worse thing I hate in windows is playing games that have no idea how much physically is available and goes on with swapping. This causes a lot of glitches in odd times and I find that silly. If I don't support swapping, then It'll enforce them to manage the memory properly. This is just one of my perspective in things that give software designers no slack in keeping things properly managed within a multi-tasking enviroment.

note that there's not only "swap files", there's also memory-mapped files that are strongly coupled with the "swap-in" mechanism. You'll also like the swap space when it comes to shutting down your laptop so you can resume applications once in the train

Solar wrote: Reality check: Self-paging sounds nice, but in 99% of all cases what the management worries about isn't performance but how cheap can it be implemented / maintained... and that's where VM (being transparent to the app) scores big time. Not even talking about the requirement to rewrite existing applications... but then again, if we'd worry about that, we wouldn't be doing our own OS projects.

Well, self-paging need not require any rewriting: simply implement the paging into a standard library, and let application override it if it wants to. Applications that don't care, need not be conserned whether paging is implemented by kernel, or the library.

If Linux suddenly started using self-paging, and you'd put a default implementation into GlibC, you could quite easily do it in such a way, that any dynamically linked binary wouldn't even need to be recompiled.

The main thing you need, is for applications to modify their own mappings, and catch paging violations in userspace. This is a useful feature even in globally paged systems, because it means paging hardware can be used to trigger garbage collections, implement write-barriers, and do other kinds of fancy stuff like catch memory bugs (Electric Fence anyone?).

Asking such an userspace fault handler to talk with a backing store to page in, isn't actually that hard to do, and then you basicly have self-paging. Which you can put into the C runtime library.

On the other hand, applications that benefit from knowing the exact size of memory available to them, can do their own special memory management, overriding the library defaults.

I don't see this being hard to implement, and since most of the required features are usually implemented for other purposes anyway, I think many if not most production operating systems could support self-paging without any kernel modifications: just disable normal virtual memory, and add a default pager into some standard library. You really want to allow application access to "accessed" and "modified" flags though, to avoid introducing too much overhead.

So summary would basicly be: If you aren't going to do global virtual memory, then providing enough functionality to let an application self-page if it wants isn't terribly hard to do. To support legacy applications that depend on virtual memory being there, adding a default pager system into a standard library solves the problem.

mystran wrote:Well, self-paging need not require any rewriting: simply implement the paging into a standard library, and let application override it if it wants to. Applications that don't care, need not be conserned whether paging is implemented by kernel, or the library.

As I see it, the problems that self-paging attempts to solve are twofold:

• Adding a swapping mechanism to a microkernel is hard.
• Apps might have enough knowledge of their memory usage that they could provide a better page replacement policy than the OS.

I think you can at least solve the second problem just by allowing apps to provide hints to the OS' VMM about which pages are unlikely to be used any time soon (I can see a generational GC providing some pretty good hints here).

The issue Singularity faces with swapping (as I see it) is that it would increase the size of the "trusted base" dramatically. One of the invariants of Singularity is that one running process cannot affect any other except by communicating with it over IPC channels -- this includes a complete ban on shared memory. But as soon as you try swapping out to disk through a disk driver (and maybe file system driver), you are exposing other processes' state to those drivers. Therefore, you have to trust them (a lot)! The same issue exists if you want to put the VMM policy outside the kernel (like in L4).

In my OS, the difficulty I have with VMM in general is that I want to put most of the VMM policy outside the ukernel to keep the kernel simple, but I also want to support fast IPC via page mapping, which doesn't work since the VMM server is supposed to be aware of which process owns which pages at all times (but the kernel itself implements IPC). I suspect that putting the VMM in the kernel is a better idea, but I want to keep my kernel as small as possible. Without swapping, that VMM policy would be much simpler and could easily go in the kernel, but I also suspect that the need for swapping is not going away any time soon.

AST's comments about swapping remind me of when he argued against multithreading 15 years ago.

I think virtual memory isn't really required on mid-range systems any more, and has never been required for limited-domain systems (where you know exactly what is running all the time, eg. an mp3 player).

For low-end desktops, I think we will see a move towards thin-client systems, or diskless systems which can run applications locally. Swapping to a server (rather than to disk) is an interesting idea to look at, but I don't know if it would be practical.

For high-end workstations and servers, virtual memory is a requirement, whether in kernel or in a library. Servers tend to have a few memory-intensive applications running at all times (eg. database app), and workstations tend to have many applications running at once with widely varying memory requirements.

I think when the day comes that you can edit a 3D landscape with tens to hundreds of high-definition textures in memory at all times, then you can forget virtual memory. I also think that day will probably never arrive as memory demand grows with memory supply.

Hi

Colonel Kernel wrote:In my OS, the difficulty I have with VMM in general is that I want to put most of the VMM policy outside the ukernel to keep the kernel simple, but I also want to support fast IPC via page mapping, which doesn't work since the VMM server is supposed to be aware of which process owns which pages at all times (but the kernel itself implements IPC). I suspect that putting the VMM in the kernel is a better idea, but I want to keep my kernel as small as possible. Without swapping, that VMM policy would be much simpler and could easily go in the kernel, but I also suspect that the need for swapping is not going away any time soon.

IMHO keeping the kernel as small as possible isn't a practical design goal unless there's actually a reason for it. For e.g. (taking it to the extreme) the kernel could consist of the GDT and IDT, some IRQ/exception stubs, one function to switch to CPL=0 (with some permission checks) and nothing else. In general, the smaller the micro-kernel becomes the harder it is to implement the OS as a whole and the worse performance gets.

So the question is, what would you gain by shifting the VMM into user space? It won't be any more reliable (the kernel couldn't prevent bugs in the VMM from messing up other processes, the VMM couldn't be restarted if it crashes, you couldn't run a redundant pair of VMMs and the system is toast if the VMM crashes regardless of where it is). It might make the kernel itself easier to write and maintain, but the end result ("kernel + VMM + interfacing") will be harder to write and maintain.


Cheers,

Brendan

Hi,

paulbarker wrote:I think when the day comes that you can edit a 3D landscape with tens to hundreds of high-definition textures in memory at all times, then you can forget virtual memory. I also think that day will probably never arrive as memory demand grows with memory supply.

When you can have two "latest release" games running on the same computer and "alt+tab" between them, then VMM won't be needed (and we'll all ride unicorns to work, even though petrol/gas will be free, because it'll be easier to avoid the flying pigs).


Cheers,

Brendan

Brendan wrote:IMHO keeping the kernel as small as possible isn't a practical design goal unless there's actually a reason for it. For e.g. (taking it to the extreme) the kernel could consist of the GDT and IDT, some IRQ/exception stubs, one function to switch to CPL=0 (with some permission checks) and nothing else.

That sounds like an exokernel to me.

In general, the smaller the micro-kernel becomes the harder it is to implement the OS as a whole and the worse performance gets.

That's debatable... L4 does a pretty good job of keeping VMM policy in user-space without sacrificing too much performance.

So the question is, what would you gain by shifting the VMM into user space?

The flexibility to implement different VMM policies for different apps in the same system. I guess you could characterize this as one of the main goals of an exokernel, but not necessarily of a microkernel. As I said myself recently, flexibility is only an end in itself if you're a gymnast.

It won't be any more reliable (the kernel couldn't prevent bugs in the VMM from messing up other processes, the VMM couldn't be restarted if it crashes, you couldn't run a redundant pair of VMMs and the system is toast if the VMM crashes regardless of where it is).

Point taken.

It might make the kernel itself easier to write and maintain, but the end result ("kernel + VMM + interfacing") will be harder to write and maintain.

That's also debatable, but the other reasons you've given sound pretty compelling. The reason I went down this road was to avoid having to implement a complex VMM right away, but I suppose as an alternative I could design the rest of my kernel so that I can plug in swapping and advanced VMM policies later on.

The fact that the kernel has to trust the swap server still bothers me on some level, but I guess there isn't much that can be done about it...