Access more memory in Real Mode?

[Jaffe]

Nope, that part actually works! I set ESI to point at msg_test, and the string gets printed on the screen perfectly. It stops _before_ the print-loop (or in the start of it). It must be somewhere in the MoveString loop, I think.

[blip]

You really can't count on that behavior, there IS code out there that does reload DS in the INT 10h handler. Also that code does not work with ESI > 65535 which is why you think it does work. The offset of msg_test is probably <= 65535 unlike 100000h. When DS gets reloaded and then restored, its value is the same but the base and limit are changed to conform to real mode rules. In order to handle this reload situation you could reset DS's base and limit every time you think they may change, or alternatively handle the GPF generated when an item with offset of > 65535 is attempted to be accessed and in that handler you reset DS's parameters. Also this way you can turn on interrupts without risking a reload. IIRC this is what FASM does.

[Jaffe]

So what you mean is that some BIOS's reset the segment limits when a BIOS-function is called?

If you look at the print routine, AL gets the value in [ESI] before it is called, so it shouldn't have anything to do with ESI pointing to a value above 65535.

[Pype.Clicker]

You really can't count on that behavior, there IS code out there that does reload DS in the INT 10h handler. Also that code does not work with ESI > 65535 which is why you think it does work. The offset of msg_test is probably <= 65535 unlike 100000h.

Do you happen to have experience with unreal mode that stopped to be unreal due to reloading a segment register ? afaik, in real mode, modifying the segment register will not affect shadowed limit and type.

Now, indeed if you call a BIOS service that do switch to pmode and back, you'll no longer be in unreal mode.

[blip]

I haven't touched unreal mode for years so I can't say I remember from experience but this seems to say that reloading a segment register will load it with real mode compatible parameters. You can't claim that there isn't some INT 10h handler out there somewhere that does:

Code: Select all

push ds   ; save DS's value
push word 0
pop ds   ; DS = 0, i.e. DS reload here
;... do some stuff
pop ds   ; restore old DS value, but with real mode rules saying what the base and limit will be

So when the BIOS returns to you, DS will have its old value but its base and limit won't be the same as before. The limit is important here, it keeps you from trying to access beyond 64K from the base. If you attempt to violate it, a GPF is raised.

[JAAman]

yes, loading the segment register will always reload the hidden portion from the GDT, except that in RMode any reference to the GDT is redirected to an internal table instead (the internal table is hard-coded to work a little differently, and holds values which simulate RMode values (the 386+ CPUs dont actually have a separate 'RMode' -- its just a special case of PMode, all checks are performed just like in 'normal' PMode, except a lot of exceptions are ignored, it uses an internal GDT, and a 16bit IDT))

this is why intel recomends doing a far jump after returning to RMode (to reload the hidden portion of CS), and why it is impractical to use a 32bit 4GB CS in UMode (the first interupt -- hard or soft -- will crash the system, due to an incompatible stack)

[Pype.Clicker]

hm. doesn't make much sense to me. First of all, i remember of TRAN's "start32" stuff reloading DS and friends with selectors to 64K segments _before returning to RMode_ for the purpose of being in rmode (not umode).

Second, say i want to have unreal mode with segment GS and FS (and thus avoid odd stuff with the bios).
I enable pmode and load GS=FS=0x18=unreal segment. Limit is cached. fine.
Let's go back to (un)real mode. What will be the base of my GS segment if i don't change it ?? 0x18 ? that'd be unpractical. Yet, i guess the only way to find out is to try out, but yet ...

looking at BabyStep (which i trust having been extensively tried):

Code: Select all

  cli      ; no interrupt
   push ds      ; save real mode

   lgdt [gdtinfo]   ; load gdt register

   mov  eax, cr0   ; switch to pmode by
   or al,1         ; set pmode bit
   mov  cr0, eax

   mov  bx, 0x08   ; select descriptor 1
   mov  ds, bx   ; 8h = 1000b

   and al,0xFE     ; back to realmode
   mov  cr0, eax   ; by toggling bit again

   pop ds      ; get back old segment
   sti

   mov bx, 0x0f01   ; attrib/char of smiley
   mov eax, 0x0b8000 ; note 32 bit offset
   mov word [ds:eax], bx

[blip]

Second, say i want to have unreal mode with segment GS and FS (and thus avoid odd stuff with the bios).

Not trying to be a nitpick but personally I wouldn't count on the assumption that BIOS or DOS code doesn't fiddle with FS or GS, even if they likely would restore it. If that doesn't phase you, consider a TSR that could hook BIOS or DOS interrupts, or even a boot program that remains resident by decrementing the conventional memory count in the BDA (I think that's where it is anyway). These could very well use all segment registers. For everyone wanting to use unreal mode I highly suggest implementing the GPF handler. If you are concerned with it being right on top of an IRQ handler you can remap the PICs or change the IVT's address and code the proper handlers. Complications complications, why can't you guys just be happy with pmode? Just kidding, unreal mode sure has its place. Pype, I will test that code if I can when I get home to settle this.

Edit: Test completed. The character was not written to the screen unless I removed the POP DS.

[Pype.Clicker]

For everyone wanting to use unreal mode I highly suggest implementing the GPF handler. If you are concerned with it being right on top of an IRQ handler you can remap the PICs or change the IVT's address and code the proper handlers.

I wouldn't remap the pic: the BIOS might just get completely confused. The nice thing is that when a GPF occurs, the cpu uses the IDT even in real mode ... of course you have no IDT in real mode and thus it will tripple fault and reset if you try to run "mov eax,[0xb8000]".

Pype, I will test that code if I can when I get home to settle this.

Edit: Test completed. The character was not written to the screen unless I removed the POP DS.

It worked on Qemu. and on my AMD K6-II system aswell. Can you give more info about the testbed you used (bios version, board manufacturer, cpu vendor and version ?)

btw, did you used just the code i posted or the independent bootsector found in the BabySteps article ?

Code: Select all

;==========================================
; nasmw boot.asm -o boot.bin
; partcopy boot.bin 0 200 -f0

[ORG 0x7c00]      ; add to offsets

start:   xor ax, ax   ; make it zero
   mov ds, ax   ; DS=0
   mov ss, ax   ; stack starts at 0
   mov sp, 0x9c00   ; 200h past code start

   cli      ; no interrupt
   push ds      ; save real mode

   lgdt [gdtinfo]   ; load gdt register

   mov  eax, cr0   ; switch to pmode by
   or al,1         ; set pmode bit
   mov  cr0, eax

   mov  bx, 0x08   ; select descriptor 1
   mov  ds, bx   ; 8h = 1000b

   and al,0xFE     ; back to realmode
   mov  cr0, eax   ; by toggling bit again

   pop ds      ; get back old segment
   sti

   mov bx, 0x0f01   ; attrib/char of smiley
   mov eax, 0x0b8000 ; note 32 bit offset
   mov word [ds:eax], bx

   jmp $      ; loop forever

gdtinfo:
   dw gdt_end - gdt - 1   ;last byte in table
   dd gdt         ;start of table

gdt        dd 0,0  ; entry 0 is always unused
flatdesc    db 0xff, 0xff, 0, 0, 0, 10010010b, 01001111b, 0
gdt_end:

   times 510-($-$$) db 0  ; fill sector w/ 0's
   db 0x55          ; req'd by some BIOSes
   db 0xAA
;==========================================

[JAAman]

say i want to have unreal mode with segment GS and FS (and thus avoid odd stuff with the bios).
I enable pmode and load GS=FS=0x18=unreal segment. Limit is cached. fine.
Let's go back to (un)real mode. What will be the base of my GS segment if i don't change it ?? 0x18 ? that'd be unpractical. Yet, i guess the only way to find out is to try out, but yet ...

no your selector will be 0x18, the hidden portion will contain whatever was loaded into it from your GDT (so your base and limit will be what you set in your GDT, and will remain so until you reload the segment selector


think of it like this:
386+ CPUs have no RMode -- rather they emulate RMode by using a specially formed internal GDT -- all the rules from PMode are still in effect, but if the segment is changed, then new hidden portion values are loaded from the GDT: but since we are in RMode, the request to the GDT is redirected to an internal table which contains values to emulate RMode addressing

this is clearly described in the intel manuals for one reason -- when the 386+ CPU is first initialized, it boots in unreal mode not RMode -- the segments are the same as the 8086 boot, but the hidden portion is set to match a 16bit data segment located at the top of phisical memory (look at vol.3 section 9.1.4 -- which clearly states that hidden portions remain in effect until a new value is loaded into the segment register) this section is very clear (esp the last paragraph) and explains why it is impractial to use 32bit 4GB CS in UMode

[blip]

My testbed was a 1.33 GHz AMD Athlon and normally runs Win98. I don't remember the BIOS version or board manufacturer but it was a PC I built some time ago with a brother. I used the code you posted, filling in the GDTR and GDT, running in real mode using Turbo Debugger 2.51. I had to enter the hex opcodes for the CR0 MOVes if it matters.

The GPF I was talking about would happen in real mode after your unreal segment parameters are wiped out and then you try to use it to access an address past the 64 KB limit. By changing the IDTR in real mode, you can change where the CPU looks for the IVT and specify how long it is. By doing this you can effectively always maintain a pre-hook on any interrupt handler since the CPU will always look at your program's IVT and not the traditional one at the first 1 KB of memory, given that the IDTR isn't loaded with something else later on. Of course you will need to write code for every handler you're not using to jump to the proper vector in the traditional IVT location. By keeping the pre-hook on IRQ 5's handler you can check the PIC to see if there is a pending IRQ 5 to decide whether the handler was called to handle a GPF or IRQ.

I'm not really seeing how the BIOS would get confused from a PIC remapping (as an alternative to IVT relocation). Just point all of the vectors to the proper IRQ handler addresses, i.e. just copying a block of far addresses from one part of the IVT to another, and then point the exception vectors to your handlers.

Do you think this stuff overkill?

[Pype.Clicker]

maybe you should try to use it straight as a bootsector as it's expected. I remind of my attempts to debug protected mode (and unreal mode, etc.) stuff with TDebug -- a good tool most of the times, but which get screwed as soon as you mess with CR0...

Btw, what's the value of DS after you popped it ? Without popping it, you end up with your character drawn at 0xB8080 (not the top-left corner of the screen, but still on the screen).
If it's popped somewhere at 0x1234, that may lead you to 0xb9234, which is no longer on the screen.

sorry if i sound over-suspicious about your experiments: i know they were unreal-mode TSR programs that did need the help of a GPF handler to work back in DOS times, but i always thought that was to prevent about "odd situations" and all the examples i've seen without such "odd situations" have been working properly on all the machines i've seen.

That's the reason why i've re-posted the babystep example completely to see what happens when we run the same code.

[asmboozer]

Code: Select all

   [bits 16]
   org 1000h

   mov esi, msg_test
   mov edi, 100000h

MoveString:
   mov al, byte [ds:esi]
   cmp al, 0
   je PrintString  

you didn't move 0 into ds:edi

Code: Select all

   mov byte [ds:edi], al
   inc esi
   inc edi
   jmp MoveString

PrintString:
   mov esi, 100000h

PrintLoop:
   mov al, byte [ds:esi]
   cmp al, 0 

how can al be zero? unless you have zeroed the memory
from 0x1000000?

Code: Select all

   je Halt
   mov ah, 0Eh
   int 10h
   inc esi
   jmp PrintLoop

Halt:   
   mov ah, 0Eh
   mov al, 'O'
   int 10h

   jmp $

   msg_test db "Just some testing...",0

[asmboozer]

segment located at the top of phisical memory (look at vol.3 section 9.1.4 -- which clearly states that hidden portions remain in effect

no such section, only 9.1 no subsection found in the section

[blip]

Pype: I haven't had much time since it's exam week, but I will test when I can. Sorry if you got the impression that I wrote DOS TSRs implementing unreal mode--I was just throwing ideas out there. From what I've read, all that stuff should work. :-\

asmboozer: The value 0 didn't need to be moved into DS:ESI because his string is pointed to by it and it is null-terminated. His code goes through the source string, character by character, and copies it to DS:EDI until it hits the zero byte (I noticed this isn't copied), at which point it prints the copy that is at 1 MB. At least that's what he intended for it to do.