Real machine vs bochs

[egos]

I meant that TS should put N into HiddenSectors field by hand because in his source this field holds 0.

[Minoto]

Ah, now I understand what you meant. Thanks -- I was thinking in terms of an existing, correct BPB.

[DavidCooper]

Now we are on to something !! No I did not account for any partitioning, i set up the USB using HxD the same way I set up a disk Image for Bochs! I am a newbie and I wasnt aware of the partition table thingy ( so that could very well explain why there are always zeros in the RAM)

So when you say it works in Bochs, you're actually working from an image rather than a real drive, and that means there's no guarantee at all that the physical disk you're using on real hardware is correctly organised. I can't actually find a way to make Bochs run a physical disk, but I have just run my OS off a physical disk in Qemu for the first time to see if Qemu can do this, and it can. So, if you can get your OS to run from an image in Qemu but it fails to run from a physical drive in Qemu, that should tell you something useful.

[Ahmed]

Hello Everyone !!
Thanks for everything so far !!
So I have done some reading and this is what I have concluded and I just need some clarification on certain things.
1) MBR is at absolute PHYSICAL (cylinder 0, head 0, sector 1 -- or, alternately, LBA 0)
2) MBR contains the partition table as well as boot code
3) There is something else called the Volume boot record which is found at the beginning of every partition available ( naturally at a certain offset from the MBR which is at the very beginning of the USB). This is only present for HDD and USB , not for floppy
4) I was treating the VBR as if it was at the location of the MBR, hence, I always read the wrong sectors.

Now what I dont understand is:

1) Will the bios auto detect the VBR and load it @ 0x7C00 instead of the MBR ?
2) After going through the standard procedure with DiskPart this is what I found @ sector 0 using HxD:

0000 EB 3C 90 4D 53 44 4F 53 35 2E 30 00 02 01 02 00 ë<.MSDOS5.0..... Sector 0
0010 00 00 00 00 80 00 29 4D B0 71 10 4E 4F 20 4E 41 ...€øø÷.?.ÿ.€...
0020 02 00 02 80 F8 F8 F7 00 3F 00 FF 00 80 00 00 00 ....€.)M°q.NO NA
0030 4D 45 20 20 20 20 46 41 54 31 36 20 20 20 33 C9 ME FAT16 3É
0040 8E D1 BC F0 7B 8E D9 B8 00 20 8E C0 FC BD 00 7C ŽÑ¼ð{ŽÙ¸. ŽÀü½.|
0050 38 4E 24 7D 24 8B C1 99 E8 3C 01 72 1C 83 EB 3A 8N$}$‹Á™è<.r.ƒë:
0060 66 A1 1C 7C 26 66 3B 07 26 8A 57 FC 75 06 80 CA f¡.|&f;.&ŠWüu.€Ê
0070 02 88 56 02 80 C3 10 73 EB 33 C9 8A 46 10 98 F7 .ˆV.€Ã.së3ÉŠF.˜÷
0080 66 16 03 46 1C 13 56 1E 03 46 0E 13 D1 8B 76 11 f..F..V..F..Ñ‹v.
0090 60 89 46 FC 89 56 FE B8 20 00 F7 E6 8B 5E 0B 03 `‰Fü‰Vþ¸ .÷æ‹^..
00A0 C3 48 F7 F3 01 46 FC 11 4E FE 61 BF 00 00 E8 E6 ÃH÷ó.Fü.Nþa¿..èæ
00B0 00 72 39 26 38 2D 74 17 60 B1 0B BE A1 7D F3 A6 .r9&8-t.`±.¾¡}ó¦
00C0 61 74 32 4E 74 09 83 C7 20 3B FB 72 E6 EB DC A0 at2Nt.ƒÇ ;ûræëÜ 
00D0 FB 7D B4 7D 8B F0 AC 98 40 74 0C 48 74 13 B4 0E û}´}‹ð¬˜@t.Ht.´.
00E0 BB 07 00 CD 10 EB EF A0 FD 7D EB E6 A0 FC 7D EB »..Í.ëï ý}ëæ ü}ë
00F0 E1 CD 16 CD 19 26 8B 55 1A 52 B0 01 BB 00 00 E8 áÍ.Í.&‹U.R°.»..è
0100 3B 00 72 E8 5B 8A 56 24 BE 0B 7C 8B FC C7 46 F0 ;.rè[ŠV$¾.|‹üÇFð
0110 3D 7D C7 46 F4 29 7D 8C D9 89 4E F2 89 4E F6 C6 =}ÇFô)}ŒÙ‰Nò‰NöÆ
0120 06 96 7D CB EA 03 00 00 20 0F B6 C8 66 8B 46 F8 .–}Ëê... .¶Èf‹Fø
0130 66 03 46 1C 66 8B D0 66 C1 EA 10 EB 5E 0F B6 C8 f.F.f‹ÐfÁê.ë^.¶È
0140 4A 4A 8A 46 0D 32 E4 F7 E2 03 46 FC 13 56 FE EB JJŠF.2ä÷â.Fü.Vþë
0150 4A 52 50 06 53 6A 01 6A 10 91 8B 46 18 96 92 33 JRP.Sj.j.‘‹F.–’3
0160 D2 F7 F6 91 F7 F6 42 87 CA F7 76 1A 8A F2 8A E8 Ò÷ö‘÷öB‡Ê÷v.ŠòŠè
0170 C0 CC 02 0A CC B8 01 02 80 7E 02 0E 75 04 B4 42 ÀÌ..̸..€~..u.´B
0180 8B F4 8A 56 24 CD 13 61 61 72 0B 40 75 01 42 03 ‹ôŠV$Í[email protected].
0190 5E 0B 49 75 06 F8 C3 41 BB 00 00 60 66 6A 00 EB ^.Iu.øÃA»..`fj.ë
01A0 B0 42 4F 4F 54 4D 47 52 20 20 20 20 0D 0A 52 65 °BOOTMGR ..Re
01B0 6D 6F 76 65 20 64 69 73 6B 73 20 6F 72 20 6F 74 move disks or ot
01C0 68 65 72 20 6D 65 64 69 61 2E FF 0D 0A 44 69 73 her media.ÿ..Dis
01D0 6B 20 65 72 72 6F 72 FF 0D 0A 50 72 65 73 73 20 k errorÿ..Press
01E0 61 6E 79 20 6B 65 79 20 74 6F 20 72 65 73 74 61 any key to resta
01F0 72 74 0D 0A 00 00 00 00 00 00 00 AC CB D8 55 AA rt.........¬ËØUª

Sector 1 ( is all zeros there is nothing there)


0400 F8 FF FF FF 00 00 00 00 00 00 00 00 00 00 00 00 øÿÿÿ............ Sector 2 (Thats the FAT of an empty Drive if I am not - mistaken)


Now usually this is what gets replaced with my boot loader ( I open the binary file I created (boot.bin) in HxD, I copy it and I paste it over where this and save). All offsets and calculations where made with reference to sector 0 (where this is found when usb was just formatted and where my code goes). Now the question is, is this the MBR and the partition table, or is this the VBR and I cant see the MBR using HxD. Also, if this is the MBR, does overwriting this makes the USB inaccessible by the BIOS, thus when I attempt to load, it returns rubbish ? Also if this is the MBR and I am not supposed to touch it , so how do I right to the VBR? Last thing, if this is the VBR (probably in my opinion, but please correct me) how do I access the MBR and see how much hidden sectors are there to account for them while translating from relative addressing to absolute LBA for the Bios to get the right sectors ??

I will get a Hex dump of the entire USB and upload it on google drive and post the link

Thanks alot guys !!

[Brendan]

Hi,

1) Will the bios auto detect the VBR and load it @ 0x7C00 instead of the MBR ?

No. The MBR's code typically searches the partition table for a partition that is marked as "active", then loads the first sector of that partition and passes control to it (essentially the VBR is "chainloaded" by the MBR).

2) After going through the standard procedure with DiskPart this is what I found @ sector 0 using HxD:

That looks like a default MBR (for floppy) or VBR (for partitioned devices) installed by Windows when it creates a FAT file system; which only really tells the user that the disk isn't bootable (e.g. in case they try to boot a non-bootable data disk). I don't know if you're displaying "sector 0" of a partition or if you're displaying sector 0 of a disk (I've never used DiskPart, but it's obvious there's no space for a partition table in there).


Cheers,

Brendan

[egos]

1) Will the bios auto detect the VBR and load it @ 0x7C00 instead of the MBR ?

It's possible for USB. See above what I said about hiding some sectors by BIOS. You should provide correct booting in both cases.

2) After going through the standard procedure with DiskPart this is what I found @ sector 0 using HxD...

To see all available sectors you should open PHYSICAL disk, not LOGICAL. Look at your dump. The partition starts at physical sector 0x80 (logical sector 0) because DiskPart put this value into HiddenSectors field. One "empty" sector between boot sector and FAT is a result of previous editing BPB+ structure by hand when you put 2 into ReservedSectors field. Bytes from disp 0x10 to 0x1F have very strange values. It looks as if they were placed not in own place

[Ahmed]

@ egos
Yeah your right !! I messed up while copying and pasting from HxD my bad !!

[Ahmed]

ok so I just formatted my USB drive (its an old 32 MB drive I found hidden somewhere in my room) using the following steps:
open cmd
>> Diskpart
>> list disk
>> select disk 1 ( in my case the usb was disk 1)
>> clean
>> create partition primary
>> select part 1
>> active
>> format fs = FAT
>> assign letter F
>> exit

and that was the result (this is an image I got with HxD of the entire USB, dont worry its just 32 MB )
open it with HxD
(USB_formatted.img)
https://docs.google.com/file/d/0B86LfhC ... sp=sharing

Next I this is what I would usually do ( which is not working on the real machine )
I would copy the boot sector I created and pasted in the very first sector I can access in HxD (not sure now what its called anymore), they copy and paste the Kernal from the windows explorer like any regular person would move files ( sorry dont know the name for just copy and paste with the mouse thingy from windows using GUI ... like no code or dd or program is involved)

The result is this (HDD16.img)
https://docs.google.com/file/d/0B86LfhC ... sp=sharing

now if you take this image and run it on Bochs it will work and will boot into real then protected mode and display a green screen with some text on it. If you copy this on a USB (by manual sector placement using HxD) and boot a real computer, it will boot and display some messages (real mode still), but then it will fail to load kernel and will ask you to push any key to warm reboot ( thats part of my program not standard error message)

now I would be very very very eternally grateful if someone would show me how to transform this image that works on bochs to one that works on the real computer.

Here is the source code for the boot sector:

Code: Select all

[bits 16]						
jmp main

;*********************************************
;	BIOS Parameter Block
;*********************************************
bpbOEM					db "ATLAM OS"
bpbBytesPerSector:  	DW 512
bpbSectorsPerCluster: 	DB 1
bpbReservedSectors: 	DW 2
bpbNumberOfFATs: 		DB 1
bpbRootEntries: 		DW 512
bpbTotalSectors: 		DW 0xFFFF
bpbMedia: 				DB 0xf8  ;; 0xF1
bpbSectorsPerFAT: 		DW 494
bpbSectorsPerTrack: 	DW 18
bpbHeadsPerCylinder: 	DW 2
bpbHiddenSectors: 		DD 0
bpbTotalSectorsBig:     DD 0
DRIVE_NUMBER            equ 7C24h
DRIVE_TYPE              equ 7C3Eh
NUMBER_OF_HEADS         equ 7C1Ah
NUMBER_OF_DRIVES        equ 7C3Fh
SECTORS_PER_TRACK       equ 7C18h
bsExtBootSignature: 	DB 0x29
bsSerialNumber:	        DD 0xa0a1a2a3
bsVolumeLabel: 	        DB "MOS FLOPPY "
bsFileSystem: 	        DB "FAT16   "

;*********************************************
;		End of BIOS PARAMETER BLOCK
;*********************************************
;************************************************;
;	Prints a string
;************************************************;
Print: 
			lodsb				; load next byte from string from SI to AL
			or	al, al			; Does AL=0?
			jz	PrintDone		; Yep, null terminator found-bail out
			mov	ah, 0eh			; Nope-Print the character
			int	10h
			jmp	Print			; Repeat until null terminator found
	PrintDone:
ret								; we are done, so return

;************************************************;
; Convert CHS to LBA
;************************************************;

ClusterLBA:
          sub     ax, 0x0002                          ; zero base cluster number
          xor     cx, cx
          mov     cl, BYTE [bpbSectorsPerCluster]     ; convert byte to word
          mul     cx
          add     ax, WORD [datasector]               ; base data sector
          ret
;************************************************;
; 			Failing to Load Proceedure
;************************************************;		
		
FAILURE:
		
        mov     si, msgFailure
        call    Print
        mov     ah, 0x00
        int     0x16                                ; await keypress
        int     0x19                                ; warm boot computer
;************************************************;
; Convert LBA to CHS
; AX=>LBA Address to convert
;
; absolute sector = (logical sector / sectors per track) + 1
; absolute head   = (logical sector / sectors per track) MOD number of heads
; absolute track  = logical sector / (sectors per track * number of heads)
;
;************************************************;

LBACHS:
          xor     dx, dx                              ; prepare dx:ax for operation
          div     WORD [SECTORS_PER_TRACK]           ; calculate
          inc     dl                                  ; adjust for sector 0
          mov     BYTE [absoluteSector], dl
          xor     dx, dx                              ; prepare dx:ax for operation
          div     WORD [NUMBER_OF_HEADS]          ; calculate
          mov     BYTE [absoluteHead], dl
          mov     BYTE [absoluteTrack], al
          ret
;************************************************;
; Reads a series of sectors
; CX=>Number of sectors to read
; AX=>Starting sector
; ES:BX=>Buffer to read to
;************************************************;

ReadSectors:
     .MAIN
          mov     di, 0x0005                          ; five retries for error
     .SECTORLOOP
          push    ax
          push    bx
          push    cx
          call    LBACHS                              ; convert starting sector to CHS
          mov     ah, 0x02                            ; BIOS read sector
          mov     al, 0x01                     	      ; read one sector
          mov     ch, BYTE [absoluteTrack]            ; track
          mov     cl, BYTE [absoluteSector]           ; sector
          mov     dh, BYTE [absoluteHead]             ; head
          mov     dl, BYTE [DRIVE_NUMBER]            ; drive
          int     0x13                                ; invoke BIOS
          jnc     .SUCCESS                            ; test for read error
          xor     ax, ax                              ; BIOS reset disk
          int     0x13                                ; invoke BIOS
          dec     di                                  ; decrement error counter
          pop     cx
          pop     bx
          pop     ax
          jnz     .SECTORLOOP                         ; attempt to read again
          int     0x18
     .SUCCESS
		  mov     si, msgReadSuccess
          call    Print
          pop     cx
          pop     bx
          pop     ax
          add     bx, WORD [bpbBytesPerSector]        ; queue next buffer
		  inc     ax                                  ; queue next sector
          loop    .MAIN                               ; read next sector
          ret

;*********************************************
;	Bootloader Entry Point
;*********************************************


main:
;----------------------------------------------------
     ; code located at 0000:7C00, adjust segment registers
     ;----------------------------------------------------
		 cli						; disable interrupts
		 
         mov     ax, 0x07C0				; setup registers to point to our segment
		 mov     ds, ax
         mov     es, ax
         mov     fs, ax
         mov     gs, ax

     ;----------------------------------------------------
     ; create stack
     ;----------------------------------------------------
     
      mov     ax, 0x0000				; set the stack
      mov     ss, ax
      mov     sp, 0xFFFF
      sti								; restore interrupts
	 ;----------------------------------------------------
	 CLD
	 
DiskParam:
	 
push dx 
push es 
mov ah, 8 
int 13h                       ; get drive parameters 
mov [DRIVE_TYPE], bl 
and cx, 3Fh                   ; maximum sector number 
mov [SECTORS_PER_TRACK], cx 
mov [NUMBER_OF_DRIVES], dl 
movzx dx, dh                  ; maximum head number 
add dx, 1 
mov [NUMBER_OF_HEADS], dx 
pop es 
pop dx 
mov [DRIVE_NUMBER], dl

LOAD:
          xor 	  cx,cx
		  xor 	  ax,ax
		  xor     bx,bx
          mov 	  ax, 0x0210     						; LBA of Sector to read
          mov     bx, 0x0440                            ; Address to Load to
		  mov 	  cx, 0x0002							; Number of sectors to read
		  call    ReadSectors
		  
	mov     cx, 0x000D                 	; Counter String length
	mov		di,0x05A3					; Expected String Location in RAM
	mov     si, String 					; String requiring matching
    repe cmpsb                                         ; test for entry match
	je      KERNL
	jmp     FAILURE
    
KERNL:
mov     si, msgConfirmString
call    Print

jmp 	0x07C0:0x0440
;*********************************************
;		Messages Section
;*********************************************
String 									db "Searching for Operating System"
msgReadSuccess 							db 0x0D, 0x0A, "Krnl Rd Suc", 0x0D, 0x0A, 0x00
msgConfirmString 						db 0x0D, 0x0A, "Kernel Confirmed in Memory", 0x0D, 0x0A, 0x00
msgReadFail 							db 0x0D, 0x0A, "Krnl Rd Fld", 0x0D, 0x0A, 0x00
msgFailure  							db 0x0D, 0x0A, "ERR:Pres Key Rbt", 0x0A, 0x00
;*********************************************
;			Data Section
;*********************************************
absoluteSector 						db 0x00
absoluteHead   						db 0x00
absoluteTrack  						db 0x00     
datasector  						dw 0x0000
cluster    							dw 0x0000
 

;*********************************************
;		Boot Sector Signature
;*********************************************

	 TIMES 510-($-$$) DB 0
     DW 0xAA55

Thanks alot guys !!

[edit by AJ: please use code tags]

[Ahmed]

oh useful addresses to check out

start of boot sector @ offset: 0x000000

single FAT copy is availabble

start of FAT @ offset: 0x00000400

start of root directory @ offset: 0x0003E000

Start of kernal (actual file) @ offset: 0x00042000

[egos]

There is no actual reason to destroy original BPB+ structure. You can not specify any actual values in your code at all. Look at my BPB+ definition technique:

Code: Select all

  org 7C00h

  virtual
  bs BS
  end virtual

  jmp short @f
  nop
  rb BS_SIZE-3
@@:
  ...
  mov [bs.DriveNumber],dl
; or
  mov [$$+BS.DriveNumber],dl
; if virtual block was not declared

Many times I was saying that original HiddenSectors value will be useful for you. Besides, you didn't consider my advices touching the code.

[Ahmed]

@ egos
Thanks alot for your advice I really appreciate it !!that was just a copy and paste of the original copy i started with just so that I can see the complete transformation.

I dont know what is the proper BPB structure (I was under the impression its only used by ... well me, during the boot, and later on any file system employed, but thats a different story).

can you please help me out a bit here

org 7C00h ---> ok no problem here

-----------------------------------------------
virtual
bs BS
end virtual is this section the BPB ??
------------------------------------------------
beyond that point I dont quite get what you mean here ... can you please break it down a bit more ?( I know I am a bit hopeless)

jmp short @f
nop
rb BS_SIZE-3
@@:
...
mov [bs.DriveNumber],dl -----for example I know that dl has the drive number and I need to save it before meesing up dx, but whats the difference between this line and the one with the $$ thingy
; or
mov [$$+BS.DriveNumber],dl
; if virtual block was not declared

also please what is a virtual block ?

Thanks alot !!

[Ahmed]

and btw I got this BPB from this broken throne tutorial ... I was under the impression that its correct, can you please give me an example of a correct BPB ( at least the layout, then will worry about the values later)

[egos]

org 7C00h ---> ok no problem here
...

It was just example. You can use a traditional technique as well:

Code: Select all

  org 7C00h
  jmp short @f
  nop
  OEMName rb 8
  BytesPerSector dw ?
  ...
@@:
  ...
  mov [DriveNumber],dl ; actually I use a stack to save local variables

The main idea is to keep original values in BPB+. Just be sure that initial jump instruction points to your entry point, and write your code into boot sector starting from disp BS_SIZE within the sector. Here is pseudo code to install FAT12/16 boot loader:

Code: Select all

move 3
pass BS_SIZE-3
move 512-BS_SIZE

The correct BPB+ structure for FAT12/16 is:

Code: Select all

struc BS
{
  .Jump rb 3
  .OEMName rb 8
  .BytesPerSector dw ?
  .SectorsPerCluster db ?
  .ReservedSectors dw ?
  .FATs db ?
  .RootEntries dw ?
  .TotalSectors16 dw ?
  .Media db ?
  .FATSize16 dw ?
  .SPT dw ?
  .Heads dw ?
  .HiddenSectors dd ?
  .TotalSectors32 dd ?
  .DriveNumber db ?
  .Reserved1 db ?
  .BootSignature db ?
  .VolumeID dd ?
  .VolumeLabel rb 11
  .FSType rb 8
}

[Ahmed]

Guys !!! I LOVE YOU ALLL !!!
It works now !!!

Will be posting the complete source and binaries sooon as soon as I clean things up, but ust for reference it was the hidden sectors!! Floppies are non partitioned so the MBR was the first thing, however, USB and HDD are partitioned devices so my code actually goes to the VBR (not the MBR) and therefore when using LBA, I had to account for the hidden values. When I used the correct BPB that egos kindly provided, It worked !!

Cheers !!