Mem above 1 MB

[AR]

You have to request the map explicitly in the multiboot header, if the map isn't available and you've set the bit in the header then GRUB will [should] refuse to boot your kernel so that shouldn't be a problem.

Bochs' BIOS doesn't work properly, it only gives you the free ranges not the used ranges (usually only 2 entries, before 1MB and after 1MB), but then again, you may come across a BIOS that does the same thing in the wild so you may as well make sure it works in Bochs.

[Freanan]

Oh, i did not notice that.
I am using the multiboot header from Bran's Kernel dev tutorial, which looks thus:

Code: Select all

; This part MUST be 4byte aligned, so we solve that issue using 'ALIGN 4'
ALIGN 4
mboot:
    ; Multiboot macros to make a few lines later more readable
    MULTIBOOT_PAGE_ALIGN   equ 1<<0
    MULTIBOOT_MEMORY_INFO   equ 1<<1
    MULTIBOOT_AOUT_KLUDGE   equ 1<<16
    MULTIBOOT_HEADER_MAGIC   equ 0x1BADB002
    MULTIBOOT_HEADER_FLAGS   equ MULTIBOOT_PAGE_ALIGN | MULTIBOOT_MEMORY_INFO | MULTIBOOT_AOUT_KLUDGE
    MULTIBOOT_CHECKSUM   equ -(MULTIBOOT_HEADER_MAGIC + MULTIBOOT_HEADER_FLAGS)
    EXTERN code, bss, end

    ; This is the GRUB Multiboot header. A boot signature
    dd MULTIBOOT_HEADER_MAGIC
    dd MULTIBOOT_HEADER_FLAGS
    dd MULTIBOOT_CHECKSUM

It looks as if this code set the memory info bit..
Still the flags that my kernel gets from grub are
1111000111
And there is not valid memory information, though grub does not refuse to load the kernel.

Edit:
I just read through the multiboot specs and it says the following:

If bit 1 in the flags word is set, then information on available memory via at least the mem_* fields of the Multiboot information structure (see Boot information format) must be included. If the boot loader is capable of passing a memory map (the mmap_* fields) and one exists, then it may be included as well.

Ie my kernel might be loaded even without a valid memory map.
So maybe it is really a problem with the bochs bios...
But what do other kernels do when they are tested in bochs?
I mean i am sure most programmers want to use grub as bootloader and they will need the memory map as i do?

[AR]

In that case there is probably a problem with your code that reads the map. Bochs' BIOS (Both 2.1.1 and 2.2) have the memory map implemented, but like I said, it's a bit dodgy.

[Freanan]

In that case there is probably a problem with your code that reads the map. Bochs' BIOS (Both 2.1.1 and 2.2) have the memory map implemented, but like I said, it's a bit dodgy.

Not that i would wonder about that, but why is the flag for the mmap set to zero as well?
Anyway, here is the code that i use (which, apart from the text output and the call to the still empty pageframes_from_region function is taken from the osfaq (unsigned long long int variant)):

Code: Select all

bool mmap_to_pagestack(unsigned long* stack, mbinfo* mbi)
{   
   if((mbi->flags & 0x04000000) == 0)
      puts("   Memory map probably broken!\n");
   
   mmap* memmap = mbi->mmap_addr;

   //Loop through the memory map
   while(memmap < (mbi->mmap_addr + mbi->mmap_length))
   {
      puts("   Memory area from 0x");
      putnum(memmap->base_addr, 16);
      puts("   to 0x");
      putnum(memmap->base_addr+memmap->length, 16);
      
      //write adresses of pages inside to stack
      pageframes_from_region(memmap->base_addr, memmap->length, stack);
      
      //next entry in the memory map
      memmap+=(memmap->size + sizeof(unsigned int));
      puts("   Next entry: 0x");
      putnum(memmap,16);
   };

   puts("   Done with the memory map.\n");
   
   return true;
};

and this is the bochs output:

Code: Select all

Welcome to the Newbie Kernel!
Multiboot-Info at 0x2D9E0
Flags: 1111000111
Lower memory: 0x27F
Upper memory: 0x7FC00
Memory map start: 0x545C4
Length in kilobytes: 48
Commandline: (fd0)/kernel.xtn

Creating GDT for flat memory.
Setting up paging.
Setting up physical memory manager.
        Memory map probably broken!
        Memory area from 0x0
        to 0x9FC00
        Next entry: 0x54804
        Done with the memory map.
Setting up IDT.
Inserting ISRs into IDT.
Setting up IRQs.

Lowlevel Steps completed successfully!

[AR]

This line is wrong:

Code: Select all

memmap+=(memmap->size + sizeof(unsigned int));

See the following example:

Code: Select all

unsigned int myarr[2] = { 23, 34 };
unsigned int *myptr = myarr;
printf("Value 1 = %d\n", *myptr);
++myptr; // Moves the pointer forward by sizeof(unsigned int) NOT 1
printf("Value 2 = %d\n", *myptr);

You need to change that line to:

Code: Select all

memmap = (mmap*) ((unsigned int)memmap + memmap->size + sizeof(unsigned int))

[Freanan]

Thanks, that also shows that there are some pointer errors in my physical memory management code...

[marcio.f]

When the pc starts (in the BIOS POST) I get these memory values:

Code: Select all

Base: 640 KB
Extended: 523264 KB
Total: 524288 KB (= Extended + 1024 KB)

whereas GRUB says:

Code: Select all

Base: 639 KB
Extended: 523200 KB
Total: 524224 KB

that's 64 KB not reported. I think 32 KB belongs to the kernel being executed, right? And what about the rest?

[Solar]

Wild guesses: GRUB stage2? Preliminary stack? Multiboot data structure?

The latter two could easily be checked for...

[Pype.Clicker]

or RAM being used internally for shadowing the bios ROM ...

[marcio.f]

GRUB stage2 file is 99 KB in size so this one can't be.. right?

In the kernel loader there's 32 KB reserved for the initial kernel stack space; now there's 32 KB left.
Using sizeof, it says the size of the multiboot data structure is 52 (bytes), is this correct?

One more question, why does GRUB only reports 639 KB of base memory? Shouldn't it be 640 KB (which is the maximum) ?

Thanks,

@Pype.Clicker
Can that really happen?

[Kemp]

Considering that RAM accesses are substantially quicker then BIOS ROM accesses, shadowing is a very good idea.

[Candy]

One more question, why does GRUB only reports 639 KB of base memory? Shouldn't it be 640 KB (which is the maximum) ?

No, it shouldn't be. Even though there is 640k available, the area at the end can be reserved (and usually is) for the EBDA. This is commonly 1 or 2 kilobytes so it probably accounts for the missing bit. Check your E820 map for more details.

[Pype.Clicker]

@Pype.Clicker
Can that really happen?

Yes, certainly. ROM/EEPROM and even Flash memory is too slow to be effectively used by the CPU when executing BIOS code, so it's common for PC system to copy the content of that ROM into some RAM that is then transparently used instead of the ROM bank through some mechanism internal to the motherboard.

Now, it actually had an impact on the amount of memory when you're considering your system to be 640K + extended memory ... on a system with 2 banks of 256MB, what occurs is more likely that memory at A0000 ... AFFFFF or B0000 .. BFFFFF and memory at C0000 ... C7FFF is used for shadowing video memory, so you might not notice something that disappeared.

The only way to find out would be to inspect memory regions that GRUB don't give you and see what's left there. Afaik, GRUB states that "a new GDT should be set up by the loaded kernel", and this is probably because both the old GDT, stage2 and probably more stuff are in the "free zones" GRUB reports when listing memory.

[marcio.f]

@Kemp
Didn't know that, thanks.

[hr]
@Candy
You're right, but as GRUB automagically enables the A20 I must return to real mode or v86 in order to check the E820 map. Even so, the multiboot structure passed by GRUB has a memory map too

Is the following code snippet ok? I mean, I tested it and seems ok, but who knows:

Code: Select all

__inline__ bool v86_mode()
{
   /* SMSW is a 286+ instruction whereas V86 mode is a 386+ feature */
   __asm__ __volatile__ ("SMSW %ax");
   return ((read_eax() & 1) == 0) ? True : False;
}

[hr]
@Pype.Clicker
I've searched a little more and found this neat table about the physical memory layout: http://my.execpc.com/~geezer/osd/ram/index.htm#layout.

As you correctly said:

linear address range   real-mode address range      memory type   use
A0000- BFFFF         A000:0000-B000:FFFF         video RAM      VGA framebuffers
C0000- C7FFF         C000:0000-C000:7FFF         ROM            video BIOS (32K is typical size)

But isn't it ?dangerous? to inspect memory regions GRUB didn't reported?

[hr]
An off-topic (maybe silly) question: can C code be run in real mode?
If yes, is this code for checking the A20 gate status correct?

Code: Select all

__inline__ bool a20_enabled()
{
   ulong *mem_1mb, *mem_base;
   
   mem_1mb = (ulong *)0x00100000; /* pointer to 0xFFFF:0x0010 (or 0x00100000): 1 MB */
   mem_base = (ulong *)0x00000000; /* pointer to 0x0000:0x0000 */
   
   /* Write back and flush internal caches.
      Flush CPU caches to be sure we're actually testing memory */
   __asm__ __volatile__ ("WBINVD");
   
   /* if they're the same, then 0xFFFF:0x0010 is wrapping
        around to 0x0000:0x0000 and the A20 gate is disabled */
   return (*mem_1mb == *mem_base) ? False : True;
}

[Kemp]

You're right, but as GRUB automagically enables the A20 I must return to real mode or v86 in order to check the E820 map

I don't see how that's a problem, unless you mean "enables protected mode"