Memory Detection >4GB

[Zenith]

I'm just wondering, is there any way to reliably detect how much memory you have above 4GB?

I'm not sure if the int 15h, ax=e820 function supports memory above 4gb, though it does return 64-bit addresses...

Also, does the memory map GRUB gives you support >4gb as well?

I'm thinking they do, but I'm not sure.

Just wondering,
Julian

[naiksidd_85]

Dont know for sure but you will have write some thing similar to hugemem kernel....as the 32 bit kernel will detect 4 gb for starting

[Brendan]

Hi,

I'm not sure if the int 15h, ax=e820 function supports memory above 4gb, though it does return 64-bit addresses...

The "Int 0x15, eax = 0xE820" BIOS function does return areas above 4 GB.

Also, does the memory map GRUB gives you support >4gb as well?

GRUB just returns whatever the he "Int 0x15, eax = 0xE820" BIOS function returns (without sorting areas in order and without any sanity checking for overlapping areas, etc AFAIK).

@naiksidd_85: Yes - you'd need to use long mode (for a 64-bit kernel); or PAE or PSE36 (for a 32-bit kernel) to access anything above 4 GB.


Cheers,

Brendan

[codemastersnake]

You can Use a for loop to detect memory. Though it's nt safe but is useful as a quick dirty algo if you can't find anything. Here's the pseudo code for it:

Code: Select all

1) Define start address fairly far enough so that kernel; image don't get damaged
2) run a loop
3) assign a value to the current location
4) read the value from the location assigned
5) if value read == value assigned then increment the ddress and the amount of memory detected
6) if value read != value assigned  then break out of the loop
7) return the detected RAM

I am using this algo for memory detection. Though I am not sure whether I should use it or not. But You can always play with OS on BOCHS [/list]

[JamesM]

That algorithm is horrible. It would detect, for example, the video framebuffers as useable RAM!

[codemastersnake]

That algorithm is horrible. It would detect, for example, the video framebuffers as useable RAM!

Yeah I know It's horrible but if you start from a address let's say from 1MB then you would skip all the essential memory locations that can cause problems.

moreover you can do it as early as possible when kernel is freshly loaded so that there';s nothing in the memory to be overwritten.

I just see above algo as quick dirty way to calculate mem.

[cyr1x]

Yeah I know It's horrible but if you start from a address let's say from 1MB then you would skip all the essential memory locations that can cause problems.

Erm... NO! What about LAPIC, IOAPIC, PCI(-E,x), etc... devices.They're all above 1MB. Writing random values to memory might damage your PC. Well that is very unlikely, but ... dont't do it.

[bluecode]

Iirc there could also be memory-mapped devices between the 15th and 16th MB.

[Brendan]

Hi,

There can be a memory mapped device from 15 MB to 16 MB (typically "VESA local bus" video cards, and older ISA cards that aren't necessarily limited to just video).

There can also be an (extremely rare) "memory hole" at 0x00080000 for some sort of compatability with ancient cards (the chipset in my Intel P4 system still supports this memory hole!).

On modern systems there can also be faulty RAM that "int 0x15, eax=0xE820" says not to use, that can be anywhere (except in the first 1 MB I assume).

There can also be large arbitrary memory holes - for e.g. a NUMA system with RAM up to 0x1FFFFFFF, a hole from 0x20000000 to 0x3FFFFFFF, then more RAM from 0x40000000 to 0x5FFFFFFF.

I'd also be careful at higher address. For e.g. it's theoretically possible for the memory controller to truncate physical addresses, so that accessing the address 0x80000123 on a motherboard that only supports 2 GB of RAM will wrap around and access RAM at 0x00000123 because the highest addressing bit/s aren't connected. In this case (for a sequential search) you won't notice the problem unless the motherboard has the maximum amount of RAM installed.

Then there's memory mapped devices (PCI video cards, HPET, PCI express configuration space, APICs, etc).

There's also (typically older) motherboards where you can write a value to "nothing" and read the same value back due to bus capacitance; and there's motherboards where you write a value to cache and read the same value back from cache even though no RAM is at that address.

There's (older, mostly 80386) motherboards that remap the RAM underneath the option ROMs and BIOS to the end of RAM. For e.g. with 4 MB of RAM installed you get RAM from 0x00000000 to 0x000A0000 and more RAM from 0x00100000 to 0x00460000, which causes problems if you test each MB of RAM because you get the wrong answer (either under-counting RAM up to 0x00400000, or over-counting RAM up to 0x00500000).

Then there can be important data structures left in RAM by the BIOS (e.g. the ACPI tables) that you'd trash.

Also, with the ("set then test") algorithm posted by Snake, you could be unlucky and incorrectly decide some ROM is actually RAM because it happened to contain the same value you're using to test with.

Some of these problems can be fixed - use "read, modify, dummy write, test", check at the end of each page instead of at each MB, disable caches, etc. Some of the problems can't be fixed.

If you do write the code properly (ie. to avoid as many of the problems as you can), then it's *insanely* slow - my code took about 15 seconds to detect 512 MB of RAM on a 1.6GHz Pentium 4. This is mostly because of RAM speeds (CPU speed doesn't really matter too much). This is going to get worse because "installed RAM" is increasing relatively quickly while RAM speed isn't. For e.g. for the same RAM probing code, a computer with 2 GB of "twice as fast" RAM would probably take about 30 seconds, a computer with 8 GB of "four times as fast" RAM would probably take 60 seconds, and a computer with 32 GB of "eight times as fast" RAM would probably take 2 minutes.

Lastly, testing for RAM (if it actually works) will only tell you where RAM is - it doesn't give you a complete map of the physical address space. You won't know where you can safely put memory mapped PCI devices because you won't know which areas are reserved for chipset things (e.g. SMM, ROMs), etc.

In contrast to this, using the BIOS functions isn't too hard, is much more reliable, gives complete information and is extremely fast in comparison.

However, there are also some (older) systems that don't support some "memory detection" BIOS functions and/or you can't tell the difference between known BIOS bugs/limits and valid values; where manually probing may be necessary as a "worst case" fallback. I've got an 80486 like this here - I use whatever the BIOS does support to limit the amount of probing done, and then use conservative code with lots of caution (e.g. assuming memory holes are present when they may not be, instead of assuming memory holes aren't present when they may be). Of course my code also tries eight different methods of getting information from the BIOS (in the hope of avoiding manual probing)...

If you want to make things easy for yourself, then just use "int 0x15, eax=0xE820" and refuse to boot if this BIOS function isn't supported. This should work fine if you don't care much about older computers (80486 and older).


Cheers,

Brendan

[codemastersnake]

Thats a good article.

But as you say that one should stick to BIOS function and forget about older PCs. Then what should one do if his OS encounters and older PC and is not able detect the amount of mem correctly and crashes.

My statergy would be:

don't run on a PC with the limited support. i.e detect the oldest compatible function that your OS uses and refuse to run on a m/c below that compatibility level.


Above was just an alternative but there should be some waqy out of it? what do ya say?