Process RAM quota

[salil_bhagurkar]

Hi...
Is it possible to assign a specific amount of RAM to a process. That means, you can set, say 20 MB for a process, and prevent the process from using more RAM even if there is RAM present...

The VM size of the process still remains the same and it can use the swap as its remaining storage. But its physical memory usage should be limited. This would enable you to prevent disk caches getting destroyed by a memory hungry process...

I face this kind of a problem when i am compiling the linux kernel and running firefox. gcc and ld seem to allocate a lot of memory and it then becomes difficult for me to switch between firefox and the compiling shell. What i want to do is assign some fixed amount of usable RAM to the shell and let firefox get more memory... I don't care much about the compilation speed...

It's possible to nice the process.. But that's only for the cpu usage. I wish to do it for my RAM too... So is this implemented?

[bewing]

Are you specifically asking how to do this in linux? Or whether it's possible to create a VMM that has per-process limits on the number of physical pages allocated?

[Brynet-Inc]

Most POSIX/SUS compliant systems use {get,set}rlimit, which you're free to use on a per-process basis, or to affect all things spawned via a shell via the "ulimit" or "limits" commands. (Typically integrated, the former for ksh/bash, the latter for csh/tcsh.. etc).

http://www.opengroup.org/onlinepubs/009 ... limit.html

On Unix systems you can also change these settings via custom login.conf classes as well...

That should answer both of your questions...

[salil_bhagurkar]

RLIMIT_DATA
This is the maximum size of a process' data segment, in bytes. If this limit is exceeded, the malloc() function shall fail with errno set to [ENOMEM].

RLIMIT_AS
This is the maximum size of a process' total available memory, in bytes. If this limit is exceeded, the malloc() and mmap() functions shall fail with errno set to [ENOMEM]. In addition, the automatic stack growth fails with the effects outlined above.

This is what the page says. Now I don't want the calls to malloc to fail. I just want to limit the physical memory. The swap still should be usable. So if i set the limit to 10MB, then a malloc after 10 MB should utilize the swap and not my physical memory...

So is this possible @Brynet-Inc

[jal]

So is this possible @Brynet-Inc

This is a forum about OS development, not about Linux how-to! Please go somewhere else.


JAL

[salil_bhagurkar]

Oops @jal, my main point was, is such a thing there in any os for that matter. Since Brynet-Inc pointed out that a feature close but not what i want, is there in linux, i just replied to it...

If such a feature does not exist, i would like to develop one.... Anyway your wish...

[jal]

Oops @jal, my main point was, is such a thing there in any os for that matter. Since Brynet-Inc pointed out that a feature close but not what i want, is there in linux, i just replied to it...
If such a feature does not exist, i would like to develop one.... Anyway your wish... :roll:

Well, I'm not aware of any OS that has this feature, but it could indeed be very useful under certain circumstances. The main difficulty being things like shared libraries and the like (determining the memory footprint of a certain application may not be easy).


JAL

[bewing]

But for the VMM just to have a list of PIDs in an array, with physical page quotas that get checked, is basically easy. It only adds a little overhead into the VMM. The VMM or scheduler would have to check the PID against the list when the process is being popped off the run queue, and then only allocate up to the limit of physical pages for it (assuming that the current number of pages is below the limit).

Or as you say, the phsyical memory page quotas can be calculated for each task based on individual task's priority. If the user wants process X to have less physical memory than process Y, then they reduce process X's priority, or raise the priority of process Y. Actually, this would probably be automatic on many OSes (such as Brendan's, I think). Lower priority tasks are not allowed to take physical memory pages away from higher priority tasks -- but not vice versa.

[jal]

The VMM or scheduler would have to check the PID against the list when the process is being popped off the run queue, and then only allocate up to the limit of physical pages for it (assuming that the current number of pages is below the limit).

Yes, but imagine that about half of the physical pages used are also used by other processes, as shared libs. Do you count them as well? And if so, swap them out, only to have them to swap back in again if the other processes are scheduled?


JAL

[salil_bhagurkar]

Shared libs... Yes that is one important point which is possibly a hinderance? But you could code to not count the shared libs. I guess pages are marked shared... So it would be easy to figure out which ones... Effectively, you don't allow the shared pages to get swapped... Thats it.. If there is a lot of mallocation then the process non-shared pages will be swapped with a higher priority and the shared ones will only be swapped when the need will grow to that point where the process would have been had it not been restricted...

[jal]

Shared libs... Yes that is one important point which is possibly a hinderance? But you could code to not count the shared libs.

Yes, but then say that many lower priority processes share a bunch of large libs, which are not used by a few higher prio processes. Then effectively they could still bother the higher prio processes, something you wanted to prevent in the first place.


JAL

[salil_bhagurkar]

Yes, so

Effectively, you don't allow the shared pages to get swapped... Thats it.. If there is a lot of mallocation then the process non-shared pages will be swapped with a higher priority and the shared ones will only be swapped when the need will grow to that point where the process would have been had it not been restricted...

You don't allow them to be swapped.. Try and swap the not-shared part of the low-prio processes... Won't that work?

[bewing]

It works until all the low-pri processes are swapped out, but what if the high priority processes need more memory than that? At some point, you have to start swapping out the shared libs, too -- so you need to give them a slightly higher priority than their "parent" processes, but you can't say that you will "never" swap them out.