[64-bit] Trouble identity-mapping lower 2 MB

[intx13]

I'm learning about long mode and following the guides on the wiki, specifically http://wiki.osdev.org/Entering_Long_Mode_Directly and http://wiki.osdev.org/User:Stephanvansc ... _Long_Mode.

Everything looks great so far and I successfully identity map the lower 1 MB of memory by filling half a page table. I can write to video memory using 64-bit registers to confirm success.

Then I try mapping the lower 2 MB of memory by filling the rest of the page table (just extending my loop). When I write anywhere between 0x100000 and 0x1FFFFF my data comes out in physical RAM between 0x200000 and 0x2FFFFF, respectively, evidenced by a Bochs RAM dump. I can clearly examine my (only) page table in RAM and see that its 256th entry is qword 0x0000000000100063 - physical address 0x100000, present, writeable, and accessed.

Why are my writes above 1 MB coming out 1 MB higher than they should be, when my writes below 1 MB work fine?

(I can attach a binary image of my PML4T, PDPT, PDT, and PT tomorrow, if necessary. I can provide the code as well, but since my tables look correct I'm not sure it would help.)

[thepowersgang]

Sounds like you have a one character bug

Please post your code (specifically the section that generates the page tables.)

I would also suggest using bochs's internal debugger, and the 'page' command to check your mappings.

[intx13]

Bummer, that means I have to recompile Bochs, which I've been trying to avoid doing. I'll give it a shot later.

Here is the loop that fills out my page table. (The PML4T, PDPT, and PDT are trivially correct.)

Code: Select all

mov EAX, 3     ; Present and read/write flags set.
mov DI, PT0    ; PT0 = 0x6000. ES = 0.
mov CX, 512    ; A full page table - 2 MB of virtual / physical memory mapped.
.iden:
  mov dword [ES:DI], EAX
  add EAX, 0x1000
  add DI, 8
  loop .iden

A RAM dump confirms that the table looks exactly like what I expect...

Code: Select all

6000 | 63 00 00 00 00 00 00 00 ; First mapping in the table: virtual 0x00 to physical 0x00. Has been accessed.
6008 | 03 10 00 00 00 00 00 00
...
6800 | 63 00 10 00 00 00 00 00 ; Here is the mapping from virtual 0x100000 to physical 0x100000. Has been accessed (by the offending write code).
6808 | 03 10 10 00 00 00 00 00
...
6FF8 | 03  F0 1F 00 00 00 00 00 ; Last mapping in the page table: virtual 0x1FF000 to physical 0x1FF000.

But a write to virtual 0x100000 puts data at physical 0x200000. Physical 0x200000 isn't the target of any mapping.

[intx13]

Alright I've recompiled Bochs and now I can use the debugger. I set up the PT as previously described and write to virtual 0x100000 with a RAM dump indicating that the data ended up at 0x200000. Bochs says...

Code: Select all

<bochs:5> info tab
cr3: 0x0000000000003000
0x00000000-0x001fffff -> 0x0000000000000000-0x00000000001fffff
<bochs:6>

So Bochs agrees that my mapping is correct. What next?

[intx13]

Ok.. I'm always hesitant to blame the system, but since I recompiled Bochs my writes do in fact go to the correct physical address. My code did not change. I'm not really satisfied with that answer though...

[Combuster]

Since you sound like you wrote your own bootloader, did you forget to enable A20 or long mode and misinterpreted the results? Otherwise, you may want to post your floppy image for reference since I get the idea that your observations are either incomplete or off: Bochs is much better tested than your code and it always has a reason for doing what it does.

[intx13]

I'm with you - I trust Bochs over myself in this context. A20 is enabled (first thing I checked) and I'm definitely in 64-bit long mode (Bochs debug command "show mode" confirms).

However, I have double-checked that the image exhibits the bug on Bochs 2.4.5 (as packaged for Ubuntu 11.04) but succeeds on the latest Bochs snapshot. The disk image is attached.

Sector 0: Contains two sections. First is assembled relative to 0000:7C00. Second is assembled relative to 0000:FD00.
Sector 1 - 62: Contains two sections, contiguous, assembled relative to 0000:0500. First is assembled for real mode. Second is assembled for 64-bit long mode.

Code: Select all

1. BIOS loads the first disk sector to 0000:7C00.
2. First section relocates the second to 0000:FD00 and jumps to it.
3. Second section...
  A. Sets up the stack to grow down from 7000:FFFF.
  B. Loads the remainder of the first disk track (sectors 1 - 62) to 0000:0500 and jumps to it.
4. Real mode code...
  A. Checks that the CPU supports CPUID.
  B. Checks for AMD64 support.
  C. Checks the A20 line and enables it if necessary.
  D. Zeroes out the PML4T, PDPT, PDT, and PT space.
  E. Reads the BIOS INT15H map into low memory.
  F. Points the PML4T to the PDPT, the PDPT to the PDT, the PDT to the PT, and identity-maps the low 2 MB in the PT.
  G. Switches to 64-bit long mode.
  H. Long jumps to the long mode code.
5. Long mode code...
  A. Sets up all selectors.
  B. Writes a marker string (qword 0xAA55AA55AA55AA55) to 0x100000.
  C. Issues the Bochs magic breakpoint.
  D. jmp $

There is additional code in the image but it's short-circuited by the jmp $.

[Chandra]

Hmm.... This comes from a person who've never implemented 'Long Mode' nor read any documentation on it. Still, from the Bochs log (I tried your image), I found that the segment registers still hold 16 bit addressing. Is that how it is supposed to be? Could that be an issue?

[intx13]

Hmm.... This comes from a person who've never implemented 'Long Mode' nor read any documentation on it. Still, from the Bochs log (I tried your image), I found that the segment registers still hold 16 bit addressing. Is that how it is supposed to be? Could that be an issue?

At the magic breakpoint you should see CS = 0x0008, DS = ES = FS = GS = 0x0010. Those are segment selectors into the GDT and select flat 64-bit long-mode segments. In long mode you can use 64-bit offsets: mov qword [ES:RDI], RAX.

[Chandra]

I know that very well. You didn't seem to understand my question at all.

[intx13]

I know that very well. You didn't seem to understand my question at all.

I guess not. Maybe you could elaborate? My GDT came from the links I provided in my first post. It's possible it should be different, although it seems to work fine. I haven't tried executing out of high memory yet.

Following up on the original issue, I'm fairly certain it's a Bochs thing. My code works just fine, except that in Bochs 2.4.5 as packaged in Ubuntu 10.4, high memory appears to be artificially physically shifted up by a meg. In the latest snapshot, everything is where I expect it to be. It could be as simple as a bug in the dump RAM feature, not the emulator itself.

The code works fine on multiple real machines as well, although obviously I can't deduce if there's any physical address shifting going on. It successfully reads the INT15H memory map and builds a contiguous virtual memory spanning all the physical memory it can find, allocating pages from the newly expanded virtual memory for new PTs, PDTs, and PDPTs as it goes. It works fine on a 4 GB machine; there's a 24 GB machine around here somewhere I want to try as a real test, as it should have a more interesting memory map.

Any other thoughts on the 1 MB wonkiness?

[Combuster]

I assume chandra is referring to the following bochs output:

01583550000i[CPU0 ] CS.d_b = 16 bit
01583550000i[CPU0 ] SS.d_b = 16 bit

Which is slightly nasty because it just prints the D bit and ignores the L bit. (and D=0=16 bits whenever L=1=long mode unless you want a #GP)

I've tried all bochs versions I had lying around and couldn't reproduce the bug. (Hint: do not use old versions )

[intx13]

I assume chandra is referring to the following bochs output:

01583550000i[CPU0 ] CS.d_b = 16 bit
01583550000i[CPU0 ] SS.d_b = 16 bit

Which is slightly nasty because it just prints the D bit and ignores the L bit. (and D=0=16 bits whenever L=1=long mode unless you want a #GP)

Gotcha.

I've tried all bochs versions I had lying around and couldn't reproduce the bug. (Hint: do not use old versions )

Yeah.. I hate blaming the host but I guess that's it. The more I think about it the more I bet it's just in the "dump ram" feature. I guess a simple test would be to try to write to the highest physical page and see what happens. If it can be read and written, it's probably the dump ram. If it can't, it's probably the emulator. Either way, that's too much effort when the latest version seems to work fine!

Thanks everybody!

[Chandra]

I assume chandra is referring to the following bochs output:

01583550000i[CPU0 ] CS.d_b = 16 bit
01583550000i[CPU0 ] SS.d_b = 16 bit

Exactly. I would've posted the Bochs dump but was too busy (noticed the short lines in my last post?) The problem was solved anyway.