Page 1 of 1

garbage collection,...

Posted: Sat May 22, 2004 2:24 pm
by Ozguxxx
Hey, I know this will be some weird topic but Im just curious. Brendan: You have told about garbage collection in some recent thread. Previously I have thought about this garbage collection stuff and I could not really make any sense out of it for the time being, I mean things are currently not that complicated for me. Why do you need garbage collection? Also
what do you mean by "to clean physical memory pages"? Again, Im just curious. It is always good to hear ideas of frontiers, so anybody is welcome to tell anything about these. Thanx...

Mods: You may want to move this message to somewhere else; well, you know it better...

Re:garbage collection,...

Posted: Sat May 22, 2004 10:36 pm
by Brendan
Hi,

Well, when I said 'garbage collection' I did so because it was easy to understand. My kernel doesn't do garbage collection in the traditional sense - it's more like 'unused physical page collection'. The kernel's data structures are all designed to allocate pages but not free them when they aren't needed, so that the kernel avoids constantly allocating and freeing pages when it's busy (it re-uses them if they are there). When the OS doesn't have much else to do it will check how many physical pages there are and (if there's not enough) free any pages the kernel isn't using.

The kernel does allocation on demand (where pages are allocated within the page fault handler in response to 'page not present'). When a new physical page of memory is needed a clean page is always allocated by the kernel (a page full of zeros). This is to prevent threads from accessing data left by the thread that had it last. It also makes address spaces look like they are full of zeroed RAM.

When a physical page is freed it's put onto a dirty page stack. When a physical page is allocated it is taken from a clean page stack. This saves doing a 'rep stosd', which uses a little CPU time (time where other processors may be waiting for the clean page stack to be available) and also helps the CPU caches - blowing away 4 Kb on an older CPU with a 16 Kb cache is avoided.

This raises the obvious question: How do pages get from the dirty page stack to the clean page stack? When the CPU doesn't have much else to do (during idle time) it cleans the dirty pages and puts them on the clean page stack :-)

It's actually a little more complicated than that though. When a dirty page is cleaned the kernel also checks that the memory is good. If the computer has faulty RAM the OS eventually finds them and doesn't use them. When the OS first boots all physical pages are put onto the dirty page stack, so that every page that is ever used has been tested and found good at least once.

Also the scheduler has 4 different sheduling policies. In policy 0 and policy 1 (high priority threads) allocated pages come from the clean page stack and go on the dirty page stack when freed as described above. Policy 3 is for idle threads, where the reverse happens - an allocated page is taken from the dirty page stack and cleaned, a freed page is cleaned and put on the clean page stack. Policy 2 is for normal threads, where it depends on how much free memory there is.

Basically the overhead of cleaning and testing physical memory is not done when there is useful work to do. During boot the idle thread/s will clean and test all physical memory while the OS is waiting for the device drivers to complete initializing (even on BOCHS)...


Cheers,

Brendan

Re:garbage collection,...

Posted: Mon May 24, 2004 1:47 pm
by kataklinger
Garbage collection is very good. For an example if you kill a process you don't have to free memory at the moment (maybe CPU has some more important job to do), so you can wait System Idle Process and free it from there.