OSDev.org

Posted: **Sun Nov 22, 2020 4:55 pm**

I'm adding a little FAT16 driver to my bootloader: I load the Root table and search for the file, then if it is found in there I load the FAT table to get information about the clusters to read (the cluster part is not yet implemented)

This is the function that I use to read the disk:

Code: Select all

;***********************************;
; Reads a series of sectors         ;
; Parameters:                       ;
;   dl => bootdrive                 ;
;   ax => sectors count             ;
;   ebx => address to load to       ;
;   ecx => LBA address              ;
; Returns:                          ;
;   cf => set if error              ;
;***********************************;
ReadSectorsLBA:
    mov [LBA_Packet.block_cout], ax
    mov [LBA_Packet.transfer_buffer], ebx
    mov [LBA_Packet.lba_value], ecx
    mov si, LBA_Packet
    mov ah, 0x42    ; Read sectors function
    int 0x13
    ret

align 4
LBA_Packet:
    .packet_size     db 0x10 ; use_transfer_64 ? 10h : 18h
    .reserved        db 0x00 ; always zero    
    .block_cout      dw 0x00 ; number of sectors to read
    .transfer_buffer dd 0x00 ; address to load in ram
    .lba_value       dq 0x00 ; LBA addres value

And this is the function that i use to load the FAT table:

Code: Select all

;****************************;
; Loads FAT table to FAT_SEG ;
;****************************;
LoadFAT:
    ; clear registers
    xor ax, ax
    xor ecx, ecx
    xor ebx, ebx
    xor dx, dx

    ; compute size of FAT and store in "ax" 
    mov ax, WORD [bpb_SectorsPerFAT]        ; sectors used by FAT

    ; compute location of FAT and store in "ecx"
    mov cx, WORD [bpb_ReservedSectors]
    add ecx, 2048                       ; temporary code for first partition

    ; read FAT into memory at FAT_SEG
    mov dl, [bpb_DriveNumber]
    mov bx, WORD FAT_SEG
    call ReadSectorsLBA
    ret

The value of ax in this case is 0x40 and it behaves like its too much: if I set ax manually to something lower like 0x30 it works. I thought it was a problem with the address in memory (witch btw is 0x500) so I tried with a different one but it still does not work.
This is what I tried:
- Initialize the disk with int 13h/ah=0h
- Changing ram destination
- implementing this code outside of the function

I also tried to print the error code from the interrupt but the next line is never executed, is like it hangs forever in the interrupt handling.
Does the interrupt have some limitation on the sectors that it can read or is just some bug in my code?

It is not the minmal example of my code since it contains also the code of the kernel that sould be loaded by this bootloader but here you can find the repository with all the code. The bootloader part is in the boot directory. The readme suggest to run the dependencies.sh file but what it mostly does is creating the cross compiler for the kernel so it can be skipped. You can use the makefile in the boot direcoty to build the bootloader bins and the create_image.sh script with some adjustments.

Posted: **Sun Nov 22, 2020 5:02 pm**

0x500 + (0x40 *512) happens to reach where the original bootloader was loaded (0x30 doesn't). Possibly you clobbered you own bootloader code if you didn't relocate it away from 0x7c00.

Posted: **Sun Nov 22, 2020 5:07 pm**

Bonfra wrote:The value of ax in this case is 0x40 and it behaves like its too much: if I set ax manually to something lower like 0x30 it works. I thought it was a problem with the address in memory (witch btw is 0x500) so I tried with a different one but it still does not work.

Reading 0x40 sectors:
Your current executing code is loaded at 0x7C00. 0x40 times 512 bytes a sector is 0x8000, loaded at 0x0500. This is a range of 0x0500 to 0x804FF. Therefore, as soon you read 0x40 sectors, it has over-written your boot loader at 0x7C00. On return from the Interrupt service, you are executing new code, which is probably garbage.

Reading 0x30 sectors:
Your current executing code is loaded at 0x7C00. 0x30 times 512 bytes a sector is 0x6000, loaded at 0x0500. This is a range of 0x0500 to 0x604FF. Therefore, reading 0x30 sectors does not over-written your boot loader at 0x7C00.

Ben
- http://www.fysnet.net/osdesign_book_series.htm

Posted: **Sun Nov 22, 2020 5:42 pm**

MichaelPetch wrote:0x500 + (0x40 *512) happens to reach where the original bootloader was loaded (0x30 doesn't). Possibly you clobbered you own bootloader code if you didn't relocate it away from 0x7c00.

Yes, it makes total sense. I don't know why I didn't thought about this, I also created an excel document to take not af all the memory use under 0x100000... But anyway 0x40 *512 = 0x8000 witch overlaps 0x7c00 even if I load it at address 0x0000. Where can I load the table without overriding any data?

This is how I planned the memory usage:

But now I have to relocate the FAT16 buffer to a valid address that does not override the bootloader.
Do you have any suggestions?

EDIT:
Testing some addresses I found out that 0x3F000 does not crush but 0x30000 does. Why?

Posted: **Sun Nov 22, 2020 6:09 pm**

A lot of the DOS bootloaders would relocate the bootloader from 0x0000:0x7c00 to 0x0000:0x0500 (some may suggest 0x0000:0x520 if you were on very ancient system). You can then place the stack at the top of available low memory below the EBDA (0x0000:0x0413 contains the size in KiB of memory from 0x0000:0x0000 to the beginning of the EBDA). Multiply that by 64 will give you the segment of the EBDA. You can compute a Stack segment below that to cover the needed stack space you think you need.

By moving your bootloader code and stack out of the way you don't need to worry about inadvertently clobbering them. Code like this could do the relocation and set up the stack of a given size below the EBDA:

Code: Select all

ORG 0x500
BOOT_ORIG_OFS   EQU 0x7c00
STACK_SIZE      EQU 1024
BOOT_SECT_SIZE  EQU 512
BDA_LOWMEM_SIZE EQU 0x413      ; Location of Low memory size in KiB before EBDA

; Stack size rounded to nearest 16 byte paragraph
STACK_SIZE_PARA EQU (STACK_SIZE+15)/16
STACK_TOP_OFS   EQU STACK_SIZE_PARA*16

bootstart:
    ; If floppy media place JMP and BPB here for a VBR

    xor ax, ax
    mov ds, ax
    mov es, ax                 ; DS=ES=0

    mov bx, [0x413]            ; Get amount of memory in KiB before the EBDA
    mov cl, 6                  ; 2^6 = 64
    shl bx, cl                 ; Multiply EBDA size in KiB by 64 to get
                               ;     PARAgraph number of the EBDA

    sub bx, STACK_SIZE_PARA
    mov ss, bx                 ; Set SS segment below EBDA
    mov sp, STACK_TOP_OFS      ; Set SP to top of stack

    ; Copy 256 16-bit WORDS from 0x0000:0x7c00(DS:SI) to 0x0000:0x0500(ES:DI)
    mov cx, BOOT_SECT_SIZE/2
    mov si, BOOT_ORIG_OFS
    mov di, bootstart
    rep movsw

    ; Jump to relocated code at 0x500 at the relocboot label
    jmp 0x0000:relocboot
relocboot:
    ; Bootloader has now been relocated to 0x500
    ; Stack is now below the EBDA

    ; Insert rest of code here.

    jmp $

TIMES 510-($-$$) db 0x00
dw 0xaa55

Doing this you have one contiguous piece of memory between 0x600 and the stack that we place just below the EBDA. You can then lay that out with far fewer hassles.

I assume you intend to read chunks of your kernel into a buffer in memory below 1MiB (The int 13h/AH=42h routines can't write directly to all of the memory above 1MiB) and then copy the buffer to memory above 1MiB a chunk at a time.

Note the size of memory from 0x00000 to the start of the EBDA is stored in a 16-bit WORD @ 0x00413 in the BDA. Usually that value is 639 (640-1KiB for a system with an EBDA of 1KiB). If you multiply the 16-bit word at 0x00413 by 64 you get the EBDA segment. If 0x00413 contain 639 then 639*64=40896 (0x9FC0 which is physical address 0x9FC00) which is the segment where the EBDA starts on such a system with a 1KiB EBDA.

Posted: **Sun Nov 22, 2020 11:54 pm**

Bonfra wrote:But now I have to relocate the FAT16 buffer to a valid address that does not override the bootloader.
Do you have any suggestions?

On a FAT16 volume, one FAT can be up to 128kiB (0x20000 bytes). If you want to load the entire thing at once, you need at least that much space in your memory map.

You could be clever about it and only load the sectors you need. For example, my own bootloader keeps only one sector of the FAT in memory at a time. Most FAT32 bootloaders work like this, since a single FAT on a FAT32 volume can be up to 1GiB.

Posted: **Mon Nov 23, 2020 2:47 am**

Octocontrabass wrote: On a FAT16 volume, one FAT can be up to 128kiB (0x20000 bytes). a single FAT on a FAT32 volume can be up to 1GiB.

I did some researches on the FAT size before planning the memory but I could have read a wrong or incomplete guide.

MichaelPetch wrote: A lot of the DOS bootloaders would relocate the bootloader

I like this option but since FAT can be very large I'll stick with Octocontrabass's one:

Octocontrabass wrote: You could be clever about it and only load the sectors you need [...] keep only one sector of the FAT in memory at a time.

I'll take a look ay your bootloader and try to implement it.

Posted: **Mon Nov 23, 2020 7:22 am**

Bonfra wrote:The value of ax in this case is 0x40 and it behaves like its too much: if I set ax manually to something lower like 0x30 it works.

Many BIOS implementations limit the number of sectors to 63 (0x3F). Loading more at once might work or might not.

Bonfra wrote:But now I have to relocate the FAT16 buffer to a valid address that does not override the bootloader.
Do you have any suggestions?

You won't need the paging tables and the FAT buffer at the same time. Why don't you use the buffer 0x10000 for the FAT loader and then later reuse it for the paging tables?

My boot loader for example relocates itself in runtime to the lowest address possible, and use everything above as a buffer.

Code: Select all

      0 - 4FFh   IVT/BIOS
   500h - 5FFh   stage1 stack
   600h - 7FFh   stage1 code (boot sector relocated from 7C00h)
   800h - 7FFFh  stage2
  8000h - 9FFFFh various buffers (first FAT buffers then paging tables)

Cheers,
bzt

Posted: **Mon Nov 23, 2020 8:03 am**

bzt wrote:Why don't you use the buffer 0x10000 for the FAT loader and then later reuse it for the paging tables?t

I don't know why but for some addresses just don't work. Trying to load just the Root directory at that address does not work, but loading the FAT does.

bzt wrote:Many BIOS implementations limit the number of sectors to 63 (0x3F). Loading more at once might work or might not.

This is not my case since for some addresses like 0x3F000 works.

bzt wrote:My boot loader for example relocates itself in runtime to the lowest address possible, and use everything above as a buffer

This could work but as pointed by Octocontrabass "on a FAT32 volume can be up to 1GiB" and sice I want to keep the possibility to update to a larger fat in the future I like the option to load only only small portions of the FAT table to inspect.

The important question is why some address works and some doesn't.

Posted: **Mon Nov 23, 2020 8:48 am**

Bonfra wrote:I don't know why but for some addresses just don't work. Trying to load just the Root directory at that address does not work, but loading the FAT does.

This makes no sense. What addresses are you talking about? The sector address or the memory address? You must have a bug, as this works for everybody else.

Bonfra wrote:
bzt wrote:Many BIOS implementations limit the number of sectors to 63 (0x3F). Loading more at once might work or might not.
This is not my case since for some addresses like 0x3F000 works.

I wasn't talking about the memory address but about the number of sectors in a single transaction. Those are completely different things.

Bonfra wrote:
bzt wrote:My boot loader for example relocates itself in runtime to the lowest address possible, and use everything above as a buffer
This could work but as pointed by Octocontrabass "on a FAT32 volume can be up to 1GiB" and sice I want to keep the possibility to update to a larger fat in the future I like the option to load only only small portions of the FAT table to inspect.

Again, relocation and loading has nothing to do with each other, they are completely different things. Furthermore, you wrote "I'm adding a little FAT16 driver to my bootloader", where 128K is enough (0x10000 - 0x30000). But it's perfectly fine for FAT32 too to load 128K of the table at once. The bigger part of the table you keep in the memory the fewer BIOS calls you need which considerably speeds up your loader.

Bonfra wrote:The important question is why some address works and some doesn't.

That's the bug you should find first.

BTW, here's a fully documented and commented source of the Win FAT32 boot sector, which locates and loads IO.SYS for example.
Another example (which I highly recommend) is alexfru's BootProg, with FAT16 and FAT32 boot sectors (uses int 13h/ah=42h and does not load the entire FAT).

Cheers,
bzt

Posted: **Mon Nov 23, 2020 9:21 am**

bzt wrote:
Bonfra wrote:
bzt wrote:Many BIOS implementations limit the number of sectors to 63 (0x3F). Loading more at once might work or might not.
This is not my case since for some addresses like 0x3F000 works.
I wasn't talking about the memory address but about the number of sectors in a single transaction. Those are completely different things.

Yes, want I meant was that if I load to 0x3F000 all the 64 sectors are loaded instead for another address like 0x30000 they are not loaded. So I think that the problem is not in the maximum amount of sectors that can be read.

bzt wrote:
Bonfra wrote:The important question is why some address works and some doesn't.
That's the bug you should find first.

I think I found something:
This is the code that I use to read the disk:

Code: Select all

;***********************************;
; Reads a series of sectors         ;
; Parameters:                       ;
;   dl => bootdrive                 ;
;   ax => sectors count             ;
;   ebx => address to load to       ;
;   ecx => LBA address              ;
; Returns:                          ;
;   cf => set if error              ;
;***********************************;
ReadSectorsLBA:
.init:
    pusha
    mov ah, 0x00   ; Reset disk function
    int 0x13
    jc .done
    popa

.readSec:
    mov [LBA_Packet.block_cout], ax
    mov [LBA_Packet.transfer_buffer], ebx
    mov [LBA_Packet.lba_value], ecx
    mov si, LBA_Packet
    mov ah, 0x42    ; Read sectors function
    int 0x13

.done:
    ret

align 4
LBA_Packet:
    .packet_size     db 0x10 ; use_transfer_64 ? 10h : 18h
    .reserved        db 0x00 ; always zero    
    .block_cout      dw 0x00 ; number of sectors to read
    .transfer_buffer dd 0x00 ; address to load in ram
    .lba_value       dq 0x00 ; LBA addres value

And the problems is in the transfer_buffer: On QEMU in some cases this works but in a real machine I have to hard code the address because it does not get the correct value.

bzt wrote: BTW, here's a fully documented and commented source of the Win FAT32 boot sector, which locates and loads IO.SYS for example.

Thanks, I'll try to modify my code based on this.

Posted: **Mon Nov 23, 2020 9:39 am**

If you showed us your code we might be able to identify the issue. Common problem on real hardware is not actually setting the segment registers properly. A lot of code makes assumptions that ES=DS=CS=0 when their bootloader starts when that may not be the case. Some hardware uses the value of 0x07c0 for CS and DS and ES could have anything in them. I assume that on real hardware this code of yours is booting non-floppy media? There has been a suggestion by others (I haven't experienced it myself) that with int 13h/ah=42h that the ES register on some real hardware needs to be set to the same value in the Disk Address Packet (DAP). I mention this because I've seen no documentation that should be the case but it isn't outside the realm of possibility that some BIOSes deviate from what is documented either because of a bug or some other reason.

One observation is this though. Your function takes `ebx => address to load to`. I hope you aren't putting the value 0x30000 or 0x3F000 in EBX. EBX is being copied into the DAP and that address is a real mode segment:offset pair. For EBX one of the value for EBX that would be appropriate for physical address 0x30000 is 0x30000000 (0x3000:0x0000 is physical address 0x30000). The segment is in the top 16 bits of the register and the offset is in the lower 16 bits. Physical address 0x3F000 would use a value like 0x3F000000 (0x3F00:0x0000 = physical address 0x3F000).

I suspect that the value in EBX is probably the problem and you are using 0x3F000 and 0x30000 . That may also explain why 0x30000 doesn't work and 0x3f000 does. EBX being 0x30000 would be 0x0003:0x0000 = physical address 0x00030 which is on top of the real mode interrupt vector table (which of course will make things fail miserably). 0x3F000 would be 0x0003:0xF000 which would be physical address 0x0F030 which isn't where you want to be reading data but is in a place that probably isn't clobbering vital code and data.

I may be entirely wrong without seeing the code so I'm just offering up the possibilities.

Posted: **Mon Nov 23, 2020 10:12 am**

MichaelPetch wrote:If you showed us your code we might be able to identify the issue

In the firs post I linked the github repo. Yes is not the minimal example of the problem but the part that i'm working on is in the boot folder.

MichaelPetch wrote:I assume that on real hardware this code of yours is booting non-floppy media?

It loads from a usb stick with burned in a HDD image.

MichaelPetch wrote:There has been a suggestion by others (I haven't experienced it myself) that with int 13h/ah=42h that the ES register on some real hardware needs to be set to the same value in the Disk Address Packet (DAP)

Could be but why if I hard code the values it works and if I set them via registers (which I double checked with GDB) it does not?

MichaelPetch wrote: I suspect that the value in EBX is probably the problem and you are using 0x3F000 and 0x30000

Sadly you suspect 100% right. I have the value 0x30000 in the ebx register and I mov it the DAP.
I can calculate the right address with:

Code: Select all

%define FAT16_ADDRESS 0x30000
%define LBA_ADDRESS (FAT16_ADDRESS  >> 4) * 0x10000

and it loads correctly in memory.

Posted: **Mon Nov 23, 2020 11:17 am**

Bonfra wrote:
Code: Select all
%define FAT16_ADDRESS 0x30000
%define LBA_ADDRESS (FAT16_ADDRESS  >> 4) * 0x10000
and it loads correctly in memory.

That will work for the specific case where FAT16_ADDRESS is a multiple of 0x10000 and that may be okay for how you intend on doing things. For any address below 0x100000 you can do a more generic version like ((FAT16_ADDRESS >> 4) << 16) | (FAT16_ADDRESS & 0xf) that will cover all addresses below 0x100000 which is what DMA was restricted to with original IBM-PC hardware. Calculating it this way is usually called the normalized form. LBA_ADDRESS might not be the best name though. Maybe FAT16_FARADDR as it is effectively an encoded FAR PTR (Segment:Offset). FAT16_ADDRESS is more like FAT16_LINADDR (a linear address).

It might be more useful to make it a macro so it can be reused:

Code: Select all

%define LIN_TO_FAR_ADDR(linaddr) (((linaddr >> 4) << 16) | (linaddr & 0xf))

and use it something like this:

Code: Select all

mov ebx, LIN_TO_FAR_ADDR(FAT16_ADDRESS)

Posted: **Mon Nov 23, 2020 12:19 pm**

I think it is worth noting, and I'm unsure if you do this or not but I will mention it anyway. It is possible to reference memory with 32-bit registers in 16-bit code like `mov ebx, 0x30000` `mov ax, [ebx]` or `mov ax, [dword 0x30000]` would move two bytes from [0x30000] to AX, but this only works if you have entered unreal mode. Some hardware may already be in unreal mode when your bootloader is run and some may not. Accessing memory beyond [0xffff] is a problem if the descriptor cache have a limit of 0xffff for the segments. QEMU takes short cuts and doesn't do all checks a normal processor does so it likely won't complain at all if you exceed the limits. This likely wouldn't be the case if you have a KVM enabled kernel and use QEMU with the `-enable-kvm` option.

OSDev.org

Does int 13h/ah=42h have sectors count limitations?

Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?

Re: Does int 13h/ah=42h have sectors count limitations?