OS Updater

[Owen]

I think I worded my response carefully. If the lid is closed and one or more processes don't signal back in time, there are plenty of things you could do. First of all, you may use sound to notify the user. If he doesn't take action in time, you could

• forcefully hibernate and notify later that things may not be reliable.
• start killing.
• act according to some configurable policy (e.g., the user might decide in advance on a per-program basis).
• etc.

Any hibernation solution would have three key requirements from me:

• Very high reliability. Apps must not be able to mess up hibernation. It must just work
• Minimum CPU usage. Pretty much every time my laptop has hibernated, it has been while the ild is closed (blocking the vents) and inside my bag (insulating it), Heat production must be minimized
• Hibernation time should be dominated by the time taken to write the suspend data to the non-volatile medium. This might involve a tradeoff in which a fast compression algorithm (e.g. LZ4) is used to reduce the quantity of data written to disk

An optimization aim should be maximum resume performance - therefore it is ideal if the OS can lazily restore applications (at the least, it should be able to get me into the foreground task within a few seconds, and then spend the next minute or so paging in the rest of the tasks on the machine)

The only time my laptop should be hibernating is when it is imminently necessary - e.g. when the battery is a couple of minutes away from being below the minimum level for RAM retention.

[Love4Boobies]

I considered compression, too. It would make sense to have a configurable policy which lets the user find the right balance between the latency for going into hibernation and the latency for waking up, while also taking into consideration things like battery life, temperature, and remaining disk space. I hadn't considered temperature; I think it's a good point but I would probably monitor it instead of deciding in advance how much power to use. Another thing to consider is what to do when the criteria can't be met---what do you leave out?

[Owen]

I considered compression, too. It would make sense to have a configurable policy which lets the user find the right balance between the latency for going into hibernation and the latency for waking up, while also taking into consideration things like battery life, temperature, and remaining disk space. I hadn't considered temperature; I think it's a good point but I would probably monitor it instead of deciding in advance how much power to use. Another thing to consider is what to do when the criteria can't be met---what do you leave out?

You wonder why you can't meet the same capabilities as my 4 year old MacBook manages

As for remaining disk space - on laptops at least, I would consider hibernation a mandatory feature, and therefore always reserve as much disk space as the machine has RAM installed on the hard drive somewhere (there may be cases where the machine has insufficient disk space and the amount of installed RAM has just been upgraded. In these cases, you should probably warn the user of this fact)

[Love4Boobies]

I wouldn't like that. The machine I'm currently typing from has 32 GiB of RAM installed and most of the time I use around 20% of it. Thus, reserving so much space seems quite wasteful. I stand by my opinion that the best way to go is to give the user the ability to decide what should happen. As an aside, what you propose comes close to a useful fault tolerance strategy: Don't just reserve the space for hibernation but instead always dump to it (I believe KeyOS' memory manager did this)---you also get hibernation for free but that's beside the point. Disadvantages include extra power consumption, higher operating temperatures, quite a bit of overhead, and increased disk usage. But if your work is worth a lot, it can be a good idea.

[linguofreak]

I wouldn't like that. The machine I'm currently typing from has 32 GiB of RAM installed and most of the time I use around 20% of it. Thus, reserving so much space seems quite wasteful.

Interestingly enough, the same is true of the machine I'm typing on. But it also has way more disk than it currently needs, so I currently have it configured with a ridiculous amount of swap (over a terabyte).

[Love4Boobies]

I only have a 1 TiB HDD and a 256 GiB SSD on it. Out of curiosity, what kind of data sets are you working with that you need that much swap space?

[Owen]

I only have a 1 TiB HDD and a 256 GiB SSD on it. Out of curiosity, what kind of data sets are you working with that you need that much swap space?

Would you miss 3% of that 1TiB HDD if it was used for hibernation space? N.B. that here I'm mostly discussing schemes for laptops, where loss of power is something one should be concerned about

[AndrewAPrice]

Would you miss 3% of that 1TiB HDD if it was used for hibernation space? N.B. that here I'm mostly discussing schemes for laptops, where loss of power is something one should be concerned about

What if power loss occurs during hibernation?

We don't have to dump all memory. As the OS, we know how much memory each process uses, so we only need to dump the process's virtual memory, rather than the entire physical memory. We may have 32GB of memory, but only 1GB of memory used by open applications. This could be done by moving the entire process into the swap file. Then on resume, unswap needed pages on demand.

If your program is using 32GB of memory at hibernation time, then expect it to take a lot of disk space when dumping to disk. There can be some tricks - like sending a "Clean Up" message to applications to tell them to release any garbage/clear their caches, because some process's garbage collector may be holding on to gigabytes of garbage.

You could do a hybrid system where you send processes a "Hibernate" message, and many serialize their own state (which would be desirable in some programs that interact with the network/hardware and want to handle hibernation/resuming to save and restore an external state), and any process that doesn't respond to the hibernate message gets its memory dumped to disk.

[mallard]

Then on resume, unswap needed pages on demand.

This generally leads to a rather poor user experience on resume, as it can be several minutes before enough pages have been unswapped to make the system feel "responsive" again. A better idea would be to unswap all pages that were written out as part of the hibernate process, or do something more intelligent (e.g. unswap the most-used pages, pages with MRU times below a certain threshold or pages belonging to currently active applications, etc.)

Alternatively, doing a sparse-dump (i.e. don't dump free pages or disk cache) of all physical memory and a full reload might make a better user experience (especially if you allocate contiguous HDD space for the dump).

[linguofreak]

I only have a 1 TiB HDD and a 256 GiB SSD on it. Out of curiosity, what kind of data sets are you working with that you need that much swap space?

I don't, thus the phrasing "ridiculous amount of swap". So far, with the 32 GiB of RAM, the machine has never touched swap (though at least 32 GiB would be necessary for hibernation, if I ever actually do that). I'll probably eventually eat into the disk space currently used for swap with LVM volumes for VMs.

[Owen]

Would you miss 3% of that 1TiB HDD if it was used for hibernation space? N.B. that here I'm mostly discussing schemes for laptops, where loss of power is something one should be concerned about

What if power loss occurs during hibernation?

We don't have to dump all memory. As the OS, we know how much memory each process uses, so we only need to dump the process's virtual memory, rather than the entire physical memory. We may have 32GB of memory, but only 1GB of memory used by open applications. This could be done by moving the entire process into the swap file. Then on resume, unswap needed pages on demand.

If your program is using 32GB of memory at hibernation time, then expect it to take a lot of disk space when dumping to disk. There can be some tricks - like sending a "Clean Up" message to applications to tell them to release any garbage/clear their caches, because some process's garbage collector may be holding on to gigabytes of garbage.

You could do a hybrid system where you send processes a "Hibernate" message, and many serialize their own state (which would be desirable in some programs that interact with the network/hardware and want to handle hibernation/resuming to save and restore an external state), and any process that doesn't respond to the hibernate message gets its memory dumped to disk.

If you're using 32GB of RAM, but there is only 16GB of free disk space, what do you do when you run out of power?

You might not use that 32GB of disk space reserved for swap/hibernation... but you need it reserved in case you do.

Asking a program to "hibernate", or "clean up" or such is great - however, what if the program tries to allocate to do so? What if you run out of memory while it tries to do so? Its' tricky, and somewhat risky.

[AndrewAPrice]

Asking a program to "hibernate", or "clean up" or such is great - however, what if the program tries to allocate to do so? What if you run out of memory while it tries to do so? Its' tricky, and somewhat risky.

That's always a risk - even with awesome compression, if the hard drive only has 1 megabyte free hibernation may be impossible.

If you want guaranteed automatic hibernation - then your only option is to have the ability to reserve disk space. Even with good compression that works in most cases, you can't be guaranteed you will never have uncompressable data.

You could have a notification if active memory > hd space: "You don't have enough disk space to hibernate. Hibernation is disabled until you close some programs or free disk space."

When I was talking about asking a program to "hibernate" itself, it's not such a far-fetched request. One of the core values of your OS (particularly UI-wise) could be persistence. If you close an application, it should try to reload as much of its state as possible when it reopens. If you close your window manager, it remembers what windows where where. If you close your word processor, it remembers what document was loaded, where the cursor was, what options were selected.

Even as a laptop user, I think there would be more benefit in focusing on 'instant-on' - a 5-10 second bootup time with near instantaneous loading of applications (that allows you to quickly resume what you were doing before) would be more useful than hibernation.

[Combuster]

If unsure, terminate app. Resource hogs should die.

[AndrewAPrice]

If unsure, terminate app. Resource hogs should die.

I agree. I hate resource hogs - especially on dumb things like shutdowns.

Shutdown should be something more along the lines of: "System is going down. You have 5 seconds to clean up or you will be terminated."

A much 'nicer' version would be to wait 5 seconds, and be given the option:
"The following applications are refusing to exit: xxx, yyy, zzz

[Turn Off Immediately] [Cancel]"

Turn Off Immediately turns the computer off immediately. No silly Windows-style shutdown routines.