OSDev.org

Kevin wrote:"Equivalent to file system corruption"... That's actually a scary thought in the context of a monolithic kernel: Any rogue pointer that would today cause memory corruption (which would be "fixed" by a reboot) could end up causing file system corruption when your whole "disk" is mapped as RAM.

The kernel doesn't have to map everything into virtual memory. I don't think there's any greater risk of file system corruption than there already is from things like buffer caches.

Rusky wrote:A lot of misunderstanding going on here.

The ability to force a clean reset has almost nothing to do with NVRAM- it just means you put all your non-persistent state (approximately everything we store in volatile RAM today) in one particular section of NVRAM that you can easily discard by deallocating it and jumping to the initialization code, which will then act as if that state had been lost (it's really still there, just treated as leftover garbage now).

So it works like this?

In the event of a power failure:
"Volatile" data and "non-volatile" data are both preserved, and the system can carry on where it left off.

In the event of a system crash:
"Volatile" data and "non-voltaile" data are both preserved (although may be in an inconsistent state) and can be recovered via a memory dump.

In the event of an intentional system restart:
"Volatile" data is wiped and "non-volatile" data is preserved so that it can be reused at the next startup without having to be rebuilt via an algorithm (e.g. routing tables, etc.).

Am I correct? If so, this sounds elegant. If not, then it's still elegant and something worth considering .

onlyonemac wrote:In the event of a power failure:
"Volatile" data and "non-volatile" data are both preserved, and the system can carry on where it left off.

In the event of a system crash:
"Volatile" data and "non-voltaile" data are both preserved (although may be in an inconsistent state) and can be recovered via a memory dump.

In the event of an intentional system restart:
"Volatile" data is wiped and "non-volatile" data is preserved so that it can be reused at the next startup without having to be rebuilt via an algorithm (e.g. routing tables, etc.).

The size of the data is important in case of a combination of NVRAM and RAM. But if there is only NVRAM, then your cases are correct. And of course, the logic behind the cases should be more carefully studied and implemented, than is the case for sleep or hibernate procedures.

embryo2 wrote:The size of the data is important in case of a combination of NVRAM and RAM. But if there is only NVRAM, then your cases are correct. And of course, the logic behind the cases should be more carefully studied and implemented, than is the case for sleep or hibernate procedures.

Yeah obviously in the case of a mixture of RAM and NVRAM one has to bear in mind
a) the size of the data
b) the performance impact of using NVRAM
c) whether or not the data is "volatile"
i.e. all "volatile" data should be stored in RAM unless RAM is full in which case it may go in NVRAM. "Non-volatile" data should always be stored in NVRAM or on disk if NVRAM is full. If RAM is available, excess RAM may be used to cache "non-volatile" data from NVRAM or from disk in order to give a performance boost, provided that changes are comitted immediately to NVRAM/disk in order to maintain system consistency at all times.

onlyonemac wrote:all "volatile" data should be stored in RAM unless RAM is full in which case it may go in NVRAM. "Non-volatile" data should always be stored in NVRAM or on disk if NVRAM is full. If RAM is available, excess RAM may be used to cache "non-volatile" data from NVRAM or from disk in order to give a performance boost, provided that changes are comitted immediately to NVRAM/disk in order to maintain system consistency at all times.

Also known as virtual memory swapping and buffer caches- which means you're just treating NVRAM as a faster disk.

Rusky wrote:which means you're just treating NVRAM as a faster disk.

Just like you treating RAM as another layer of a cache. NVRAM can give us yet another layer for the caching strategy elaboration.

A lot is being made of the 'no need to reboot''ness of NVRAM.

Do we imagine the registers or stack or belt or whatever inside the CPUs is going to be NV too?

We've had decent generally-available NVRAM for a long time already - chances are you've got it in your computer already, and its called SSD. It happens that its been far easier to treat it as 'disk' rather than RAM so as to make existing programs faster without changing paradigms.

Faster NVRAM than SSD e.g. memristors or 3D XPoint will be very welcome as faster disk. It just makes fsync() a no-op and suddenly real workloads that require durability just go magically much faster.

The next step will be for the DBMS to know that the mmapped file is on NVRAM and to do 'on-disk' sorts and such so as to avoid extra copies between RAM and NVRAM.

I'm trying to think what paradigms could change deeper in the kernel, though, and so far .. not so much. NVRAM is disk that may not need page buffers and so on, but those are small tweaks internal to the FS driver and not something that makes you change anything else much...

I've actually made a computer like that as a hobby project - 12.5MHz custom CPU, with 4Kb of volatile fast RAM and 64Kb of non-volatile (but only half as fast) FRAM.
I think i mentioned it here - http://forum.osdev.org/viewtopic.php?f=15&t=27702

I always wanted to have a computer that can be turned off, then turned on and resume working from the same place.
Didn't quite get there, but it was fun.

Resetting to a known state is not a problem - just make the bootloader check for a reset button being pressed at power up.

The OS architecture was more or less transactional. Every inactive task stays where it is, active task gets a power-fail exception.
The idea was that on power-off the volatile stuff gets copied over to the FRAM (there was a low voltage detect interrupt and a few milliseconds worth of capacitors).
In practice that does not work beyond saving the task state, and i had to assume that whatever is in the volatile memory is gone.

On re-power, a bootloader checks for magic numbers, then calls the kernel's resume entry point. The kernel re-initializes the peripherals (which are volatile), and continues the execution.

All in all, once you out of the bushes it's not much different from a regular computer on the user level.
And there really isn't any advantages (that i found) to the non-volatility.
At least not on this scale - the whole thing can be cold-booted just as fast, and most of the time is spent (re-)initializing the peripherals.

However, debugging in such a system can be FUN with a capital F.
Uninitialized variables, loose pointers and similar things can produce mind-twisting effects if you ever forget that a reset means nothing.
The closest word to it is "haunted".
The system just feels haunted.
You have to exorcise it every now and then by a full reboot.

Well, a computer with 512 GB of non-volatile RAM would be much like a computer with no RAM but a very fast hard disk.
The RAM would have a filesystem and serve as a HDD.
Data such as process memory would simply be stored inside a swapfile.

StartOS wrote:Well, a computer with 512 GB of non-volatile RAM would be much like a computer with no RAM but a very fast hard disk.
The RAM would have a filesystem and serve as a HDD.
Data such as process memory would simply be stored inside a swapfile.

That approach actually works for my OS design. I am managing RAM like a filesystem, where the lines between RAM and disk storage (and, also, NVRAM if it was present) are blurred and they are all treated equally as merely different places that "objects" can reside. Of course, objects are specified to be either volatile or non-volatie, and in the latter case the OS is obliged to keep them on non-volatile media even if they are also cached in RAM (the details of which caching system I am still working out).

I am managing RAM like a filesystem, where the lines between RAM and disk storage (and, also, NVRAM if it was present) are blurred and they are all treated equally as merely different places that "objects" can reside.

That sounds awfully familiar.

Although, rather than trying to use hardware to make non-volatile hard drive space look like RAM, I'm planning on letting the application decide when to use RAM, when to use hard drive space, and when to let the OS decide. However, reading and writing to all three will probably work the same way.

That's the plan anyway.

SpyderTL wrote:Although, rather than trying to use hardware to make non-volatile hard drive space look like RAM, I'm planning on letting the application decide when to use RAM, when to use hard drive space, and when to let the OS decide. However, reading and writing to all three will probably work the same way.

I am leaving it up to wherever the object is created or moved to. So a thread creates an object and specifies what device to store it on and where in that device's object tree (although the UNIX filesystem concept is good, I decided not to implement such a design in my OS). A storage device in turn has a set of flags, one of which states if the device is "persistant" (i.e. non-volatile storage). It is up to the thread, or the user, to decide if it is appropriate for the object to be stored on a "persistant" device, or if it is OK for it to be stored on a "non-persistant" device (which, at this point, would be in RAM). (It is also worth noting that RAM is going to be implemented as multiple devices, at this stage I'm thinking one for the kernel+loaded modules and another for "userspace" (i.e. a scratchspace for loading running applications and any temporary data that they create).)