in memory without a map

[zloba]

I've read the How do I determine the amount of RAM? article, and I got some questions. I want to have the question of memory "well and trully sorted out" before I do anything with it.

1.

WE DISCOURAGE YOU FROM DIRECTLY PROBING MEMORY

Use BIOS to get a memory map, or use GRUB.

What if I'm in protected mode (booted by GRUB), and I get *NO* memory map? (that is the case with my 486)
I only get the *size* of the extended memory. (from CMOS or multiboot)
If GRUB failed to make a map, I doubt I can do better.

- Should I just assume that all of that is usable RAM?
- Should I probe it?
- Can the 386, 486 have upper-memory-mapped stuff to watch out for? (ISA?)
- Is there a proper way to detect it?
- Can this happen with a Pentium? (which has APIC, at least)

I'd like to keep supporting 386/486, if possible.

2. (out of curiosity)

Why did I mention 64mb of memory? Because the CMOS can only hold values up to 99mb.

Where does that "99" number come from? I see that CMOS has 2 bytes for "extended memory", that's 64M=64K*1K...

[nick8325]

To 1, I think GRUB calls the BIOS function to get the memory map, and if that fails falls back to asking it how much extended and conventional memory it has. So if GRUB can't get a memory map, it probably means the BIOS doesn't support that interrupt.

The best way then is just to use the multiboot information's mem_lower and mem_upper fields, so you can use memory from 0 to mem_lower KB, and 1MB to (mem_upper KB + 1MB). The multiboot spec (Multiboot standard) says:

The value returned for upper memory is maximally the address of the first upper memory hole minus 1 megabyte. It is not guaranteed to be this value.

So there'll be no holes in the memory that GRUB returns. I don't think it's possible to do better than that, since other OSes will just ask the BIOS for the amount of extended memory as well.

I don't think you should probe it, because it'll be usable. There doesn't seem to be any need to.

I think some old cards sometimes have things mapped from 640KB to 1MB, but it depends on the particular machine - you should just ignore any memory you don't know about.

I think number 2 is a mistake, but I'm not sure...

[AR]

I think some old cards sometimes have things mapped from 640KB to 1MB, but it depends on the particular machine - you should just ignore any memory you don't know about.

No, every PC compatible x86 has this for compatibility with DOS, the things in that space are the video RAM, video BIOS and the System BIOS without which it wouldn't be possible to boot the system.

The BIOS functions automatically exclude holes, of course they are much less sophisticated then the memory map being able to only return 2 RAM extents but the memory reported should be valid, unfortuantely if the system has the 16MB hole then GRUB will only report 15MB of RAM despite the BIOS functions offering information about available RAM on the other side of the hole. There is no way to remedy this with GRUB as far as I can see either, you'll have to drop back to realmode and use INT15 E801 to query the RAM yourself if you want a reliable value.

[Brendan]

Hi,

I've got a 80486 like this too. I use the following BIOS functions:

int 0x15, eax = 0xE820
int 0x15, ax = 0xE881
int 0x15, ax = 0xE801
int 0x15, ah = 0xC7
int 0x15, ah = 0x88

None of them are supported.

I only get the *size* of the extended memory. (from CMOS or multiboot)
If GRUB failed to make a map, I doubt I can do better.

- Should I just assume that all of that is usable RAM?

I'm not sure where GRUB gets it's information.

For the values from CMOS, I wouldn't assume it's all usable RAM. The problem here is that some ISA devices map their own memory into the physical address space (for e.g. a video card's linear frame buffer). The ISA bus is limited to 16 MB and nothing on it can access memory above 16 MB, so the ISA device will use the area from 15 MB to 16 MB, overwriting the RAM that was there.

Despite this, you can probably get away with using this RAM, because most ISA devices don't map their own memory into the physical address space, but it will depend on which ISA devices are plugged in.

For my OS, the memory detection code that uses the CMOS and manual probing ignores the area from 15 MB to 16 MB. I figure it's better to be safe than sorry, and most computers that don't support any of the BIOS functions are probably too old to have more than 8 MB anyway.

- Should I probe it?

If the code you use to probe takes into account problems like bus float and caching, and if all of the BIOS functions aren't supported, then probing can work reliably. When I wrote my memory detection routines I worked from oldest to newest (starting from probing and working my way up to int 0x15, eax = 0xE820). After each one I tested it, and can say that my memory probing code works on all 6 of the computers I tested it on (ranging from 80486 to Pentium 4).

My manual probing code can be found at:
http://bcos.hopto.org/latest/sys/i386/k ... probe.html

- Can this happen with a Pentium? (which has APIC, at least)

I haven't found a Pentium that doesn't support at least one of the BIOS functions (but that doesn't mean it's impossible).

I'd like to keep supporting 386/486, if possible.

IMHO the problem with 80386 is that half of them used "expanded memory", which uses a bank switching thing (where 64 KB chunks of RAM are swapped in and out of memory below 1 MB). The other problem is they don't support the INVLPG instruction or native FPU exceptions (the NE bit of CR0). Now, it's difficult to find someone who kept an 80486 (the number of people with an 80386 would be very small). You may want to compare the difficulty in supporting them properly, and how much you'll gain if you do....

2. (out of curiosity)

Why did I mention 64mb of memory? Because the CMOS can only hold values up to 99mb.

Where does that "99" number come from? I see that CMOS has 2 bytes for "extended memory", that's 64M=64K*1K...

You're right - I'm not sure where the "99" came from either (perhaps someone thought it was BCD?) - consider it a typo.


Cheers,

Brendan

[Colonel Kernel]

I'm not sure where GRUB gets it's information.

For the values from CMOS, I wouldn't assume it's all usable RAM. The problem here is that some ISA devices map their own memory into the physical address space (for e.g. a video card's linear frame buffer). The ISA bus is limited to 16 MB and nothing on it can access memory above 16 MB, so the ISA device will use the area from 15 MB to 16 MB, overwriting the RAM that was there.

Despite this, you can probably get away with using this RAM, because most ISA devices don't map their own memory into the physical address space, but it will depend on which ISA devices are plugged in.

Argh... Why do you have to take something so simple and complicate it with reality?

AR already mentioned the hole at 15 MB:

The BIOS functions automatically exclude holes, of course they are much less sophisticated then the memory map being able to only return 2 RAM extents but the memory reported should be valid, unfortuantely if the system has the 16MB hole then GRUB will only report 15MB of RAM despite the BIOS functions offering information about available RAM on the other side of the hole.

So... Given that I don't know where GRUB gets its information either, can I nevertheless safely assume that the memory from 0 to mem_lower and from 1 MB to (1MB + mem_upper) is usable RAM?

Please answer yes or no, or my head will explode. ;D

[AR]

So... Given that I don't know where GRUB gets its information either, can I nevertheless safely assume that the memory from 0 to mem_lower and from 1 MB to (1MB + mem_upper) is usable RAM?

From 0x500 (IVT, BIOS Stack and data segment) to mem_lower and 0x100000 to mem_upper, yes. You can assume the memory is safe as much as you can trust the BIOS not to be wrong (It's possible, but it is highly unlikely that INT15 88 would be broken since it so important).

[Brendan]

Hi,

Argh... Why do you have to take something so simple and complicate it with reality?

I do this to give you enough information to decide for yourself.

It would be easy to prevent the OS from using RAM from 15 MB to 16 MB and waste 1 MB on some computers (and allow for those ISA cards). It would also be easy to use the RAM from 15 MB to 16 MB and have an OS that doesn't support or allow for those ISA cards. Either way is perfectly sane IMHO.

I guess it would also be possible to not use the RAM from 15 MB to 16 MB until after all ISA devices have been detected and configured - this would give the best results, but would be very difficult to do reliably.

The BIOS functions automatically exclude holes, of course they are much less sophisticated then the memory map being able to only return 2 RAM extents but the memory reported should be valid, unfortuantely if the system has the 16MB hole then GRUB will only report 15MB of RAM despite the BIOS functions offering information about available RAM on the other side of the hole.

So... Given that I don't know where GRUB gets its information either, can I nevertheless safely assume that the memory from 0 to mem_lower and from 1 MB to (1MB + mem_upper) is usable RAM?

Please answer yes or no, or my head will explode. ;D

YES - you can safely use it all!

The problem is that the BIOS may report less than all RAM - for example you could have a computer with 128 MB of RAM where the BIOS only reports 64 MB or 16 MB (or maybe 15 MB, and even no extended memory at all - see below).

BTW I checked out GRUB's memory detection code. All it does is get the values from the BIOS and give them to you. GRUB only uses the following BIOS functions:

int 0x15, eax = 0xE820
int 0x15, ax = 0xE801
int 0x15, ah = 0x88
int 0x12

In addition it doesn't check for some known BIOS bugs, like "E801" returning zero in AX and BX (and using CX and DX instead), or "E801" and/or "88" returning with carry set. For the "E801" BIOS bug, this will cause GRUB to report no extended memory at all, regardless of how much is installed.

I also checked out the Linux memory detection code. It's uses the same BIOS functions as GRUB, handles "E801" a little differently (and avoids the bug mentioned), and does check if "E801" returns carry set (but not "88").

Both Linux and GRUB never do manual probing and never look at the CMOS values. Of course they both allow the user to tell them they are wrong too - the 'uppermem' command for GRUB, and the 'mem=' kernel parameter for Linux).

I tend to trust the user less than I trust the BIOS (which isn't that much), so I'd rather detect memory completely and (hopefully) correctly, and not give users the option of changing it.


Cheers,

Brendan

[zloba]

Thanks for the explanation, everyone
As for the 15-16M hole, I'll probably skip it, to be safe. If a machine has >=16M, the loss of 1M is not the end of the world anyway.

From 0x500 (IVT, BIOS Stack and data segment) to mem_lower

Do I need to preserve these things (or anything below 640K), if I'm not planning to return to real mode, or use Virtual8086 tasks? (I may have to use these tasks for video, though)
Just curious - again, I'll skip that region, no big loss.

>I'd like to keep supporting 386/486, if possible.
IMHO the problem with 80386 is that half of them used "expanded memory", which uses a bank switching thing (where 64 KB chunks of RAM are swapped in and out of memory below 1 MB). The other problem is they don't support the INVLPG instruction or native FPU exceptions (the NE bit of CR0). Now, it's difficult to find someone who kept an 80486 (the number of people with an 80386 would be very small). You may want to compare the difficulty in supporting them properly, and how much you'll gain if you do....

Yeah, I guess it is too much trouble for little gain. At least 386 - they are freaking slow, too.
It's more cost- and everything-efficient to buy an old Pentium for cheap.

Would be a pity to dump my 486, tho. I have it all set up for testing, and setimental value.. It's stuffed with 32M ram and 2 NICs, and I actually used it a few years before, as fileserver/router/terminal. Wasn't too bad at these tasks.
I may be able to use it again, with a new OS

[Brendan]

Hi,

Do I need to preserve these things (or anything below 640K), if I'm not planning to return to real mode, or use Virtual8086 tasks? (I may have to use these tasks for video, though)
Just curious - again, I'll skip that region, no big loss.

You can always use RAM from 0x00000600 to the beginning of the EBDA (if any) or 0x000A0000 (if there's no EBDA) - use BIOS int 0x12 to find out where the top of usable memory below 1 MB is.

If you don't intend to use real mode or virtual 8086 mode after boot (including real mode APM functions to shut down the computer), then you can use from 0x00000000 to the top of usable memory below 1 MB.

In this case, if you also don't intend to use Intel's MP specification tables or ACPI's tables, then you could probably wipe the EBDA too (i.e. use the entire 640 KB). It can be used by other devices and things (for boot information) so I probably wouldn't do this in case you need the information one day.

IMHO it's probably better to keep the first 4 KB and the EBDA (if present) in tact. It'd be very rare for this to be over 8 KB, and means you can change your mind later.

Would be a pity to dump my 486, tho. I have it all set up for testing, and setimental value.. It's stuffed with 32M ram and 2 NICs, and I actually used it a few years before, as fileserver/router/terminal. Wasn't too bad at these tasks.
I may be able to use it again, with a new OS

There's nothing wrong with the 80486's!

An 80486 supports everything that a Pentium or later CPU does, except for features that are detected with CPUID.

Given that (IMHO) good software should always check if any of these features are present before using them, supporting 80486 adds virtually no extra hassles.

The real problem is ISA devices (which are more common on 80486 and Pentiums), and the crappy external local APICs that (SMP) 80486's use.

Deciding whether to support ISA is a completely different question (there are 80486's with PCI and Pentium 4's with ISA). I won't be supporting multi-CPU on 80486's (too rare/not worth the hassle).


Cheers,

Brendan

[Candy]

... and Pentium 4's with ISA).

.......

talk about wait cycles....

Are you sure that there practically are p4's with ISA?

[Xardfir]

Hiya,
I quote:
Microgram Computers Online Store
https://secure.mgram.com.au/default.asp

Cat No: 17086-13 Industrial ATX Mother Board P4 w 4PCI 2ISA Slots
   This industrial motherboard is equipped with the latest Intel 875P chipset, the board supports the Intel Pentium 4 Prescott processor with hyper-threading capabilities, processor speeds up to 3.2GHz with 800MHz FSB and up to 4Gbyte DDR 266/333/400 in dual channel mode.
Other special features of the board includes 256-level watchdog timer, DiskOnChip socket, digital I/O (4 in / 4 out) and PCI to ISA bridge, IDE1, IDE2 2 x SATA. Measuring 305mm by 244mm, it also has an AGP port, 4 PCI slots, 2 ISA slots, 4 onboard COM's ports. With ISA slots on board and a long life cycle, it is the platform of choice for industrial applications.
$AU 631.95

Someone must really love their ISA cards!!!

[Pype.Clicker]

Someone must really love their ISA cards!!!

If you were spending $100K in developing an ISA robot-controlling card 10 years ago and the robot hasn't changed, you wouldn't feel like spending $500K to develop a PCI equivalent of your board nowadays (not talking about the risks of ending up with a less performant / more buggy controller here )

[tom1000000]

Hello,

IMHO I think you are wasting your time developing an OS for a 486. Pentium II / 440BX motherboards can be bought for practically nothing and guess what - they support the memory map.


My OS requires a CPU that supports FXSAVE FXRSTOR and also provides a memory map. Anything less and a short message will politely request the user to upgrade their computer.


Developing for a 486 - you may as well develop your OS for a 286 or Commodore64.

[Candy]

Someone must really love their ISA cards!!!

If you were spending $100K in developing an ISA robot-controlling card 10 years ago and the robot hasn't changed, you wouldn't feel like spending $500K to develop a PCI equivalent of your board nowadays (not talking about the risks of ending up with a less performant / more buggy controller here )

Good point. Second point, do you want to support those devices / those computers?

Developing for a 486 - you may as well develop your OS for a 286 or Commodore64.

If somebody does decide to develop an OS for the C64, I can test it for you on the dutch version if you give me a tape image.

;D