Detecting memory size

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
Cemre

Detecting memory size

Post by Cemre »

mov esi , ( 1 << 20 )
; Direct probe the memory
.do until, dword [esi] , eq , dword 0xFFFFFFFF
???; till we start reading the pull-up resistors.
???add esi , ( 1 << 20 )
???.if esi , eq , 15 * ( 1 << 20 )
??????; Skip the Memory hole
??????add esi , ( 1 << 20 )
???.endif
.loopit

"esi" here is memory size
what is wrong with detecting memory size with the code above using direct probing? shouldn't the system report an 0xFFFFFFFF at a non-existent memory area? some systems seem to lock down here in my Cemre operating system ( mine works perfect ) when trying to read from a non-existent memory area.

is there another way of detecting memory size? i don't want bios, i'm in pmode, besides it has 64MB limit.

does anybody know how to read from SPD memory info? or DMI info?

PS: dont worry about the code piece, i have the macros to compile it in nasm.
durand
Member
Member
Posts: 193
Joined: Wed Dec 21, 2005 12:00 am
Location: South Africa
Contact:

Re:Detecting memory size

Post by durand »

Direct probing is dangerous because of memory-mapped devices. It is possible that you are writing into/reading from a memory mapped device and altering the registers or data of another hardware device. Especially if you are doing this at 1MB intervals.

I once saw a "map" of safe areas to directly probe but it's generally not a good idea and I don't recall where it is.

The best best is to ask the BIOS. ( I know you don't want it ). It gives you the full memory map (not just 64 MB) and it provides the ACPI memory regions, free memory regions and reserved memory regions.

You must be getting into pmode somehow.. I use GRUB to pass the memory map to my kernel. Maybe you can modify your boot loader to pass you the memory map before you enter pmode.
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re:Detecting memory size

Post by Brendan »

Hi,
Cemre wrote: what is wrong with detecting memory size with the code above using direct probing? shouldn't the system report an 0xFFFFFFFF at a non-existent memory area?
Aside from incorrectly detected memory mapped devices (as already mentioned), there's no guarantee that non-existant areas will return 0xFFFFFFFF. Some motherboards may return 0x00000000, and some could return any value at all (perhaps depending on which song the local radio station was playing at the time). To avoid this, you should write a value and see if that value was stored correctly, but there are problems with this...

On some motherboards (typically 80486 and older, but nothing prevents the same problem on new computers) the bus may be left floating. Writing a value on this floating bus can result in the value being stored by the capacitance between bus lines, such that non-exists memory areas tend to behave as if there's RAM installed. Doing a dummy read or write to settle the bus will help prevent this.

Further, testing at 1 Mb increments isn't so good either - I've seen computers that map the RAM underneath the ROMs, etc at higher addresses, so that with 2 Mb of memory you get RAM from 0x00000000 to 0x000A0000 and from 0x00100000 to 0x00260000 (an extra 384 Kb). In this case you might detect that there's RAM at 0x00200000, and attempt to use non-existant memory up to 0x002FFFFF.

Also some motherboards have a memory hole from 15 Mb to 16 Mb (and lots have a BIOS option for it that someone may have enabled). Not sure what this hole was for, but if the hole is present your code will only detect 15 Mb and stop (when there's actually more).

In general use the BIOS. The BIOS "knows" all of the intricate details of the motherboard, chipset, etc.

Now for something really stupid - some older computers (namely 80486 and possibly early pentium) don't support any of the normal extended memory size BIOS functions! On these computers you must directly probe (or simply don't support them, it is a rare enough problem now).

Therefore, the method I recommend is:

Code: Select all

  Try: int15 ax=0xe820, GET SYSTEM MEMORY MAP
     If int15 ax=0xe820 worked return
  int12, GET CONVENTIONAL MEMORY <1 Mb
  Try: int15 ax=0xe801, GET MEMORY SIZE FOR >64M CONFIGURATIONS
     if int15 ax=0xe801 worked return
  Try: int15 ah=0x88, GET EXTENDED MEMORY SIZE (286+)
     if int15 ah=0x88 worked return
  Try: Manual Probing
Please note that for all of these BIOS functions there have been buggy BIOSs that behave badly. Check each BIOS function in "Ralf Brown's Interrupt List" to see what it should do and what it might do. It is possible to write code that avoids the problems with these buggy BIOSs.

Now, if you must manually probe there are things you can do to minimize the problems.

First, test at the end of each 4 Kb (not at the start of each 1 Mb). This avoids any problems with strange physical memory mappings.

If your testing gets above 2 Gb then you've got problems - there will be no computers with this much memory that don't support "int15 ax=0xe820".

Do the scan in 2 seperate parts. One from 1 MB to 16 MB, and another from 16 Mb up to compensate for the memory hole possibility.

Then the actual test should be something like:

Code: Select all

   mov eax,[gs:edi]         ;Get original value
   not dword [gs:edi]         ;Invert memory
   wbinvd               ;Make sure memory is inverted
   not eax               ;Invert original
   mov ebp,[gs:esi]         ;Read from 0 to settle bus
   cmp [gs:edi],eax         ;Is it the same?
   jne .doneManual            ; no
   not dword [gs:edi]         ;Invert memory back to original state
I forgot to mention the CPU caches so far - it's possible to write a value to non-existant memory and read that same value back reliably, because it'll be stored in the CPU's cache/s. You have to either disable the caches (CD flag in CR0) or use "wbinvd" to flush them.

Another reason for using the BIOS is that manual detection (when done correctly) is very slow - especially on modern computers with lots of RAM when the cache can't be used effectively...


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
smiddy

Re:Detecting memory size

Post by smiddy »

Brandon hit just about everything. There is one other way of finding RAM, that is by reading the CMOS. It is only reliable to 64MB however and should be resorted to last after BIOS fails to identify the amount of RAM needed for your system.
Xardfir

Re:Detecting memory size

Post by Xardfir »

At the risk of incurring the wrath of the "Bootloader Brigade" :)

Use GRUB as your boot loader, it does extensive testing and returns the amount of memory to your OS code.

GRUB tutorial:
http://my.execpc.com/~geezer/osd/boot/grub-how.txt

Latest version is 0.95
http://www.gnu.org/software/grub/

If you *really* want to, you can 'rip' the code from GRUB and insert it into your code - make sure to distribute the GNU license with your OS though.
User avatar
Pype.Clicker
Member
Member
Posts: 5964
Joined: Wed Oct 18, 2006 2:31 am
Location: In a galaxy, far, far away
Contact:

Re:Detecting memory size

Post by Pype.Clicker »

Xardfir wrote: If you *really* want to, you can 'rip' the code from GRUB and insert it into your code - make sure to distribute the GNU license with your OS though.
<solarbot>And that you provide full sources with your bootloader (at least) which will fall under the GPL ...</solarbot>
distantvoices
Member
Member
Posts: 1600
Joined: Wed Oct 18, 2006 11:59 am
Location: Vienna/Austria
Contact:

Re:Detecting memory size

Post by distantvoices »

hihi ... solar's got a bot. The lad 's got de la bonne chance I dare say.

concerning the topic: I'd do the memorycounting per pedes only as a last fall back. Bios provides quite some calls to try ere one does this complicated and errorprone thing.

stay safe :-)
... the osdever formerly known as beyond infinity ...
BlueillusionOS iso image
durand
Member
Member
Posts: 193
Joined: Wed Dec 21, 2005 12:00 am
Location: South Africa
Contact:

Re:Detecting memory size

Post by durand »

I was thinking....

If you do a PCI enumeration, ACPI scanning (is there such a thing?), IDE,SCSI,etc scanning... and find out the mapped memory regions for each device present, then if you exclude those memory regions from the direct probe, things should be fairly safe?

It's risky. You have to know how to scan alll buses and how to query all devices.

Think it will work?
Cemre

Re:Detecting memory size

Post by Cemre »

I think I have fixed my bug,
it now uses the bios, if not supported
does a safe direct probing, see my web page..

PS: guys... thank you very very very much, this forum
has helped me far more than i ever thought about OS
programming...

www.ug.bcc.bilkent.edu.tr/~engino
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re:Detecting memory size

Post by Brendan »

Hi,
durand wrote: If you do a PCI enumeration, ACPI scanning (is there such a thing?), IDE,SCSI,etc scanning... and find out the mapped memory regions for each device present, then if you exclude those memory regions from the direct probe, things should be fairly safe?
It'd be safer if the bus scanning worked perfectly on all computers, but it'd also need to be constantly maintained to account for newer hardware standards. For e.g. when Infiniband reaches 80x86 (and here I mean direct motherboard support, not those PCI based add-on cards) you'd have to add code to handle it.
durand wrote: It's risky. You have to know how to scan alll buses and how to query all devices.

Think it will work?
Normally an OS will detect memory, set up the basic protected mode environment, and then scan buses & start device drivers, etc. You'd have to change this order (scan the buses before you configure memory management), which may be difficult to fit into the ~640 Kb conventional memory area. Then you'd probably need completely different code to scan buses once memory management and the basic protected mode environment is configured (for hot swap devices, USB devices, etc).

Using the BIOS's physical memory size functions would be much easier, and it should be impossible to write a "safer" method.


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Post Reply