Boot Loader Tutorial Idea
Re: Boot Loader Tutorial Idea
A little bit of a hint to anyone wanting to boot from CD-ROM or a flat ISO-9660 image (like you "capture" on Virtual PC to simulate a CD):
Yes, you definitely CAN boot from CD with NO floppy emulation. I've heard people say a few times you can't, but you can. My bootloader does it now. Most commercial OS's ship on a disc, and have to install before any drivers are in place. It's relatively trivial once you realize how it works. I sat around like a total dummy until I realized I was making the ISO image totally wrong.
Basically, you need an ISO image maker/CD burner which will let you set these options:
1) No floppy emulation
2) Load segment 7C0
3) Load *4* sectors
You heard right: (4 * 512) = 2048
Pad the bootloader.bin with zeros all the way to *2046* and insert the boot sig.
It's feasible with that extra space you can do all of your init in one stage and far jump right into a 32 or 64-bit kernel. I'm still going with a multi-stage though. ISOLINUX.asm is an excellent example of this, and provides quite a bit of extra code to handle problems. I strongly suggest studying it if you're interested in making your own or doing a hack'n splice job on it. If more detailed information is needed, just pm or request. I thought at first I was the odd-ball interested in this, but I guess quite a few others are too!
Yes, you definitely CAN boot from CD with NO floppy emulation. I've heard people say a few times you can't, but you can. My bootloader does it now. Most commercial OS's ship on a disc, and have to install before any drivers are in place. It's relatively trivial once you realize how it works. I sat around like a total dummy until I realized I was making the ISO image totally wrong.
Basically, you need an ISO image maker/CD burner which will let you set these options:
1) No floppy emulation
2) Load segment 7C0
3) Load *4* sectors
You heard right: (4 * 512) = 2048
Pad the bootloader.bin with zeros all the way to *2046* and insert the boot sig.
It's feasible with that extra space you can do all of your init in one stage and far jump right into a 32 or 64-bit kernel. I'm still going with a multi-stage though. ISOLINUX.asm is an excellent example of this, and provides quite a bit of extra code to handle problems. I strongly suggest studying it if you're interested in making your own or doing a hack'n splice job on it. If more detailed information is needed, just pm or request. I thought at first I was the odd-ball interested in this, but I guess quite a few others are too!
There are two major products that come out of Berkeley: LSD and UNIX. We don't believe this to be a coincidence. - Jeremy S. Anderson
Re: Boot Loader Tutorial Idea
Hi,
For anyone interested in doing this, last year I wrote the El-Torito and iso 9660 wiki pages which were just stubs at the time. They are still by no means complete, but there's some sample code and a mkisofs command line for creating the images. Any help in completing the articles much appreciated!
Also, I may be wrong, but do you need a boot sig at the end of the El-Torito boot sector? I've never used one and neither real hardware, Bochs, Qemu or VirtualPC have ever complained.
Cheers,
Adam
For anyone interested in doing this, last year I wrote the El-Torito and iso 9660 wiki pages which were just stubs at the time. They are still by no means complete, but there's some sample code and a mkisofs command line for creating the images. Any help in completing the articles much appreciated!
Also, I may be wrong, but do you need a boot sig at the end of the El-Torito boot sector? I've never used one and neither real hardware, Bochs, Qemu or VirtualPC have ever complained.
Cheers,
Adam
Re: Boot Loader Tutorial Idea
Here is the update with all the info from isolinux and the changes as suggested:
If someone could please confirm/review STEP 7 (I know the old linux boot code did this.. but i'm not sure if A) its necessary anymore and B) may actually damaged emulated FDD boots?
Once we're all happy with whats here sofar I can move on to the EDD vs CHS loading and preping for loading stage 2 from hd/emulated-hd etc.
If someone could please confirm/review STEP 7 (I know the old linux boot code did this.. but i'm not sure if A) its necessary anymore and B) may actually damaged emulated FDD boots?
Once we're all happy with whats here sofar I can move on to the EDD vs CHS loading and preping for loading stage 2 from hd/emulated-hd etc.
;=============================================================================================================
INTRODUCTION
;=============================================================================================================
A boot sector is usually the first sector of either a physical disk or partition.
The primary boot loader is 512 bytes long and MUST end with a magic signature to allow the BIOS to check
that it is bootable. The signature is:
db 055h,0aah
Our objective is to create a single unified 512 byte first stage boot loader that can be deployed to any
modern boot device and work as planned.
The first thing we need to understand and identify is what are these devices and what are the subtle
differences between them? (IE: BIOS bugs, incompatabilities or generally poorly followed standards).
Is it possible to have a single boot-sector that works for all devices?
The list of bootable devices we wish to support are:
- Real Floppy Disk
- Real Hard Disk
- USB Flash Stick (under FDD emulation from BIOS)
- USB Flash Stick (under HDD emulation from BIOS)
- CDROM/DVDROM (either natively, FD image emulation or HD emulation - refer to El Torito Boot Spec.)
USB boot is achieved through emulation. The BIOS will either assume the USB Stick to be a floppy or hard-disk
and emulate accordingly both in terms of what is passed into the boot-loader in the DL register as well as the
INT 13h disk functions. It is also possible that USB sticks will appear as ZIP, ZIP100, LS-120
or LS-240 drives but this is something we're not interested in supporting and can ignore.
Some BIOSes (most new ones i've tested) are capable of booting from USB there are however several pre-requisites:
#1 the BIOS must support it and not have any bugs.
#2 the USB stick in question must be bootable (probably a USB2.0 stick - some specifically state that they're bootable)
#3 some BIOSes that boot USB under FDD emulation specifically require there to be a BPB / FAT table at the beginning
of the boot-sector/code.
For this reason I have listed it below even though in my design it's completely redundant.
The file system I plan to use doesn't work like a traditional FS and as such I make the following assumptions:
1) For the tutorial that we'll ignore partitions and MBR and simply load the required sectors from disk.
2) There is no manadatory requirement to store the OS loaders or kernel binary within any file-system
as long as you can locate it and read it into memory.
I chose to store these linearly at the beginning of whatever device is in question and the relevant file
system information comes directly after that.
3) Because I don't use/need the BPB and my boot code is intended to be unified it will be present even in the
boot-sector of a hard-disk/cd etc.
; For reference we'll refer to this as the memory map during the stage 1 boot-loader:
; Initial Memory-Map after POST (Power on self test).
;
; 0000:0000 - 0000:03FF [ IVT Interrupt Vector Table ]
; 0000:0400 - 0000:04FF [ BIOS Data Area 40h:0000 ]
; 0000:0500 - 0000:7B00 [ Free use memory ]
; 0000:7B00 - 0000:7C00 [ Boot Loader Stack ]
; 0000:7C00 - 0000:7DFF [ BIOS Boot Load Area ]
; 8000:0000 - EBDA [ if you're paranoid you shouldn't load above 8000:0000 in case the ebda is up here ]
; [ i normally find EBDA to be around 0x9fc0:0000, max 4kb? but 1kb here as it hits vid mem ]
; 0000:7E00 - 9000:FFFF [ Free use memory ]
; A000:0000 - B000:FFFF [ VGA Video Memory ]
; C000:0000 - F000:FFFF [ BIOS ROM Memory Area ]
; FFFF:0010 - FFFF:FFFF [ Free use memory - A20 enabled ]
; FEC00000 - FFFFFFFF [ motherboard BIOS, PnP NVRAM, ACPI etc ]
;=============================================================================================================
STEP 0 (BPB table for compatability with USB FDD emulated booting)
;=============================================================================================================
org 0h
start:
;-----------------------------------------------------------------------------------
; Some BIOSes look for this BPB/BS to enable USB-FDD emulation, we'll never use it again...
; -> We'll cheat and use some of it's entries as our variables to save space.
;-----------------------------------------------------------------------------------
jmp short begin
nop
OEM DB "VXOS "
m_sectorsize DW 512 ; We're using this as our sector size variable.
Sectors_per_cluster DB 1
Reserved_sectors DW 1
Number_of_FATs DB 2
Root_entries DW 224
Total_sector_count DW 2880 ; 80*2*18
Media_descriptor DB 0F0H ; Floppy
Sectors_per_FAT DW 9
Sectors_per_track DW 18
Heads DW 2
m_cylinders DD 80
m_sectors DD 18
bsDriveNumber DB 0
Unused DB 0
bsExtBootSignature DB 29h
m_heads DD 2
bsVolumeLabel DB "VXOS "
bsFileSystem DB "FAT12 "
;=============================================================================================================
STEP 1 (Load Address Normalization)
;=============================================================================================================
Some BIOSes will load the bootsector (512bytes) to 07c0:0000 while some will load it to 0000:7c00.
In fact the BIOS may load it to any combination of seg:ofs that is equivalent to 0x7c00 linear.
The flat binary assembler source will have been created to be at a specific address (seg:ofs combination):
org 0h (assuming the segment is 07c0)
or
org 7c00h (assuming the segment is 0 and the offset is 7c00)
Either way you need to ensure this is always fixed.
In my first stage loader I work with a segment and the offset at 0.
So org 0h in the source and then a hardcoded far jump to force the segment to set:
db 0eah
dw offset cs_correction, 07c0h
cs_correction:
;=============================================================================================================
STEP 2 (Stack location choice and setup)
;=============================================================================================================
Setup the stack (I chose to keep it just below the initial load address of the boot loader).
xor ax,ax
mov es,ax ; Set ES = 0000.
mov ss,ax ; Configure Temporary Stack Just below Bootloader.
mov sp,(7c00h-16)
;=============================================================================================================
STEP 3 (Obtain boot drive number)
;=============================================================================================================
The possible values the BIOS will give us are:
00h (First floppy)
01h (Second foppy)
80h (First hard disk)
81h (Second hard disk) and so on..
Store the drive number that the BIOS passes us in DL. We must ensure DS is valid and = to CS.
mov m_driveNo,dl
Here we can also determine if we were booted from a native CD/DVD. This can be accomplished due to the fact
that the sector size of a native/non-emulated CD is 2048 bytes (2kb). As such this entire 2kb sector would have been
loaded. If we'd created our image as 2kb and stored that to disk we'd have the normal boot signature at 512 bytes
and our own dword(32bit) signature at the end of the 2kb sector. We could read the dword at memory address
0x83FC and check for the signature and we'd know that the BIOS had loaded the full 2kb sector.
An alternative approach used in ISOLINUX is to use INT13 (Refer to Ralph Brown Int list B).
mov ax,4B01h ; Get disk emulation status
mov dl,[DriveNumber]
mov si,spec_packet
int 13
INT 13 - Bootable CD-ROM - GET STATUS
AX = 4B01h
DL = drive number
DS:SI -> empty specification packet (see #00281)
Return: CF clear if successful
CF set on error
AX = return codes
DS:SI specification packet filled
Format of Bootable CD-ROM Specification Packet:
Offset Size Description (Table 00281)
00h BYTE size of packet in bytes (13h)
01h BYTE boot media type (see #00282)
02h BYTE drive number
00h floppy image
80h bootable hard disk
81h-FFh nonbootable or no emulation
03h BYTE CD-ROM controller number
04h DWORD Logical Block Address of disk image to emulate
08h WORD device specification (see also #00282)
(IDE) bit 0: drive is slave instead of master
(SCSI) bits 7-0: LUN and PUN
bits 15-8: bus number
0Ah WORD segment of 3K buffer for caching CD-ROM reads
0Ch WORD load segment for initial boot image
if 0000h, load at segment 07C0h
0Eh WORD number of 512-byte virtual sectors to load
(only valid for AH=4Ch)
10h BYTE low byte of cylinder count (for INT 13/AH=08h)
11h BYTE sector count, high bits of cylinder count (for INT 13/AH=08h)
12h BYTE head count (for INT 13/AH=08h)
The ISOLinux boot code makes a special exception for broken Award BIOSes in which apparently the interrupt
entry for int 13h is not correct and the above function will always fail resulting in a CF set. Their approach
to fix this is as follows:
Scan the BIOS code in memory for the following code pattern:
0b8h,1,2,0bbh,0,7ch,0b9h,6,0,0bah,80h,1,09ch,09ah
which is equivalent to:
mov ax,0201h
mov bx,7c00h
mov cx,0006h
mov dx,0180h
pushf
call <direct far>
We begin the scan for this string in the BIOS ROM Memory area at f000:0000.
Assuming we're scanning via ES:DI , then ES:DI+0eh will contain the updated dword value to use as the int 13h
vector and update 0000:[13h*4] with this value. If the value is the same as the original vector
3.1
If we couldn't find the award code pattern/string we can call 4801h with all disk numbers from 80-ffh.
If one of these succeed we can check that the packets returned drive number is = to the one queried and assume
we're good to continue. If they're different check if the drive numebr we're querying is the same as the original
passed into the loader in DL and once again assume we're good to go.
Award BIOS 4.51 apparently passes garbage in sp_drive member of the above packet.
but if this was the drive number originally passed in
DL then consider it "good enough"
3.2
Intel Classic R+ computer with Adaptec 1542CP BIOS 1.02
passes garbage in sp_drive, and the drive number originally
passed in DL does not have 80h bit set.
The isolinux solution now is to OR in bit 80h, then compare to the original drive and we're good to go if they're
the same. If not then loop back to 3.1
If after all this we still have nothing we're have a particularly rubbish BIOS which may not even implement function
4b01h. In this case check that the drive numebr is AT LEAST 81h and continue, else fail the boot.
Another thing to mention at this point as implemented in ISOLINUX is the following:
Some CD-ROM BIOSes have been found to corrupt segment registers and/or disable interrupts
For this reason they don't call int 13h directly but rather via a function which saves all segment registers/flags.
;=============================================================================================================
STEP 4 (Transfer code location and jump)
;=============================================================================================================
Move the first stage loader to a safe location and continue execution.
The reason I previously selected to use 07c0:0000 instead of the offset model is that it makes this
step a bit easier. I can copy the entire boot-loader code (512 bytes) to a new segment address
and then use a label directly after the far jump as the same offset for that jump (as the labels
offsets relative to the segment remain unchanged, this applies to variables etc as well).
I chose 0060:0000 as the destination (this is the address used by DOS 1.x apparently) and from looking
at a memory map after boot this space should be safe. This is only really necessary if you plan on using
the 1st stage loader as an MBR, but it doesn't hurt to do it anyway for our unified approach.
xor si,si ; DS:SI = begin of original bootloader (7c00:0000).
mov di,600h ; ES:DI = destination (0000:0600).
mov cx,100h ; Copy 256 words (512 bytes).
rep movsw ; Copy bootloader.
db 0eah ; Hardcode force far jmp.
dw offset bootloader_exc0,0060h ; new code addr = (0060:bootloader_exc0).
;---------------------------------------------------
; Bootloader will commence from here at new address.
;---------------------------------------------------
bootloader_exc0:
;=============================================================================================================
STEP 5 (Ensure segment registers are setup).
;=============================================================================================================
Ensure the data segment register correlates to CS.
push cs ; CS and DS = 0060h.
pop ds
;=============================================================================================================
STEP 6 (Determine code path based on drive number)
;=============================================================================================================
In theory, if all emulations work correctly we only need to know if the boot drive number (DL)
was >= 80h or could be masked with 80h.
That way we can use two pieces of code to load the 2nd stage loader which should support
FDD/USB emulated FDD/CDROM FDD image/DVDROM FDD image
and HDD/USB emulated HDD/CD emulated HD.
;--------------------------------------------------------------------------------
; Did we get booted from a floppy, harddisk, usb or cdrom?
; -> USB will emulate a HDD boot (80h) or FDD (00h). same for CD-ROM (ElTorito).
;--------------------------------------------------------------------------------
cmp m_driveNo,80h
jae skip_not_floppy
;=============================================================================================================
STEP 7 (Preparing for loading stage 2 when using a FD or emulated FD)
;=============================================================================================================
All FD's have 2 heads, 80 cylinders and 512 byte sectors. The only value that can vary is the sectors per
track (SPT).
Based on some of the earlier linux boot code it appears that in some machines/BIOSes have issues with
with their DPT (Diskette Parameter Table) as well as not knowing the maximum SPT value for the given media.
The following code can be used to kludge the SPT value in the Diskette Parameter Table
; The Interrupt Vector Table (IVT) occupies the first 1024 bytes of low memory and
; contains 256 4-byte entries. Each entry is an address to an Interrupt routine.
; The INT 1E vector (Diskette Configuration pointer) is at 0000:0078 and contains
; the address to the Diskette Parameter Table (DPT). (78h/4 = 1Eh)
;-----------------------------------------------------------------------------------
; -> Reset FDC Param Tables (apparently some BIOS have too low max sector
; count (7).. we need 36 for max 2.88 disks, this messes up multi-sector
; reads greater than the BIOS value. Too high is safe, too low not...
;-----------------------------------------------------------------------------------
push ds
mov bx,078h ; ES:BX = FDC BIOS Param Table Ptr.
lds si,es:[bx] ; DS:SI points to Param Table.
mov byte ptr ds:[si+4],36 ; Kludge Sector Per Track Count (36 = 2.88 disk).
mov byte ptr ds:[si+9],0fh ; Reset Head Bounce Time.
pop ds ; Updates INT 1E table.
mov dl,m_driveNo
xor ax,ax
int 13h ; Reset Controller for booted drive.
We then need to probe the disk by trying various read sizes to determine the SPT:
The disksizes or SPTs are: 36, 18, 15, 9
and these correspond to 2.88Mb / 1.44Mb / 1.2Mb / 720kb disks.
;--------------------------------------------------------------------------------
; Probe Disk Drive Sectors per Track.
; -> Try 36, then 18 and so on to see what works as BIOS has no definite way
; -> of knowing using INT13H, SERVICE 08h... more arcane rubbish.
;--------------------------------------------------------------------------------
mov ax,07c0h
mov es,ax
mov si,offset disksizes
probe_loop:
xor eax,eax
xor cx,cx
mov al,ds:[si]
inc si
cbw ; Extend AL to AX.
mov m_sectors,eax
cmp si,(offset disksizes)+4
jae short got_sect_size
mov cl,al
xor dx,dx
mov dl,m_driveNo
xor bx,bx ; Use future load address to overwrite (no danger).
mov ax,0201h
int 13h
jc short probe_loop ; If failed try next sector size down.
got_sect_size:
;=============================================================================================================
STEP 8 (Loading stage 2 loader for HD, emulated HD or native CD/DVD)
;=============================================================================================================
skip_not_floppy:
Re: Boot Loader Tutorial Idea
Ok i've finished another little bit of update to the document.
I'm going to add it to the wiki if possible and we can continue to edit/build it there.
Apparently I've just found out another strange issue with usb booting / emulation that some bioses emulation only works up to LBA 100, there-after not. So you'd need to keep a usb driver somewhere in that space (or have the whole
sys setup occur in under 100 sectors). Anyone confirm this?
Also does anyone have any input regarding the last post with sections 3/7?
I'm going to add it to the wiki if possible and we can continue to edit/build it there.
Apparently I've just found out another strange issue with usb booting / emulation that some bioses emulation only works up to LBA 100, there-after not. So you'd need to keep a usb driver somewhere in that space (or have the whole
sys setup occur in under 100 sectors). Anyone confirm this?
Also does anyone have any input regarding the last post with sections 3/7?
Re: Boot Loader Tutorial Idea
I still have severe problems with a bootloader tutorial that ignores partitioning. I simply don't think it's appropriate.
Hard drives are partitioned. Ignoring partitions is no better than making a floppy-disk-only tutorial.
Hard drives are partitioned. Ignoring partitions is no better than making a floppy-disk-only tutorial.
Every good solution is obvious once you've found it.
Re: Boot Loader Tutorial Idea
Solar I disagree, I doubt there is a single person on here (most likely) who has rolled there own multi-stage boot loader that works on as many machines as is realistically/theoretically possible.
The reason for this is that we have many tutorials covering the absolute basics and some of the various options re: partition/mbr vs. floppy but between all of us who have gone through this process we each
have specific little bits of knowledge, kludges, tweaks, bios bugs and hints and tips that have not been compiled into a single resource.
My idea is to get us all to contribute what we know and have found through references, looking at code and our own work into one single tutorial on pre-kernel startup.
I'm more than happy if you want that we could have it split into "tracks" , one for uni-boot model like mine and one for mbr/partitions and adjust as needed for various devices.
Not wanting to sound b1tchy, but everyone always seems to have complaints about others and posts/suggestions rather than actually putting our combined knowledge together to produce the most useful reference possible.
Not only would that make the process for others just starting that much easier (possibly saving them months of debugging and searching to find arcane little bits of info) but may also help us fill in gaps in our own systems.
In the medium term I'd like to see this tutorial even cover things like lists of mainboards and their issues, how to determine if your RTC is present and measure it's drift... who knows the list is very long of things that could be
built into it. It would almost be a book on it's own all about x86/x86-64 pre-kernel OS development.
Hopefully we could even at some point collaborate on some more advanced areas like h/w level programming of some specific sound cards/video boards like the intel stuff.. who knows.
We have a big forum here full of very knowledgeable people, but unless we work together I think that the majority of our efforts will not go as far as we would like, and new people might as well accept that "dabbling" is about as far as their venture is likely to go without the support and reference that we could in theory provide when our heads are all put together.
In any event, if you're keen to continue/contribute to this tutorial->eventually book/resource that would be great. If no one is I'll kill it here and we can all just carry on
The reason for this is that we have many tutorials covering the absolute basics and some of the various options re: partition/mbr vs. floppy but between all of us who have gone through this process we each
have specific little bits of knowledge, kludges, tweaks, bios bugs and hints and tips that have not been compiled into a single resource.
My idea is to get us all to contribute what we know and have found through references, looking at code and our own work into one single tutorial on pre-kernel startup.
I'm more than happy if you want that we could have it split into "tracks" , one for uni-boot model like mine and one for mbr/partitions and adjust as needed for various devices.
Not wanting to sound b1tchy, but everyone always seems to have complaints about others and posts/suggestions rather than actually putting our combined knowledge together to produce the most useful reference possible.
Not only would that make the process for others just starting that much easier (possibly saving them months of debugging and searching to find arcane little bits of info) but may also help us fill in gaps in our own systems.
In the medium term I'd like to see this tutorial even cover things like lists of mainboards and their issues, how to determine if your RTC is present and measure it's drift... who knows the list is very long of things that could be
built into it. It would almost be a book on it's own all about x86/x86-64 pre-kernel OS development.
Hopefully we could even at some point collaborate on some more advanced areas like h/w level programming of some specific sound cards/video boards like the intel stuff.. who knows.
We have a big forum here full of very knowledgeable people, but unless we work together I think that the majority of our efforts will not go as far as we would like, and new people might as well accept that "dabbling" is about as far as their venture is likely to go without the support and reference that we could in theory provide when our heads are all put together.
In any event, if you're keen to continue/contribute to this tutorial->eventually book/resource that would be great. If no one is I'll kill it here and we can all just carry on
- Combuster
- Member
- Posts: 9301
- Joined: Wed Oct 18, 2006 3:45 am
- Libera.chat IRC: [com]buster
- Location: On the balcony, where I can actually keep 1½m distance
- Contact:
Re: Boot Loader Tutorial Idea
Well, if you don't want comments, don't ask for them. But seriously, if you post a tutorial that mentions IDE, SCSI, and whatnot, but is absolutely incapable of running on them, you will seriously confuse people.
The only thing worse than no information is incorrect information.
The only thing worse than no information is incorrect information.
Re: Boot Loader Tutorial Idea
I didn't say I don't want comments, and this isn't really about me and what I want, It's about all of us contributing to one source (that is complete and accurate). If you think something I've put down is junk, then lets change it
As I was saying to Solar, I disagree with him that a tutorial that doesn't cover MBR is useless because it may fill in blanks that none of the other tutorials cover,
but I'm more than happy that we compile a resource that covers all the options - so on that front I agree with him.
Once again I'll re-iterate it, I don't know everything about this.. I just know the bits I've tried and that have worked for me, so for everyone else please feel free to write new steps/paragraphs/change the order.. whatever as long we move
towards the goal!
Been quite busy at work so haven't had much time to update.. but here is the latest with a few changes moving it more towards a tut from a series of point...
Step 4 needs work
Step 8 I''m still trying to determine if the DPT kludge is necessary and if the SPT probe is necessary AND if the probe may work for real FDD but cause issues with USB-FDD.
Perhaps step 8 can fall away and be combined with the work of 9 for HDD modes...
Another thought is that it might make sense to make the entire loader 2kb in size, and have the normal 512 byte signature where it must be, and ANY specialized CD-Native code etc can come after that.
As I was saying to Solar, I disagree with him that a tutorial that doesn't cover MBR is useless because it may fill in blanks that none of the other tutorials cover,
but I'm more than happy that we compile a resource that covers all the options - so on that front I agree with him.
Once again I'll re-iterate it, I don't know everything about this.. I just know the bits I've tried and that have worked for me, so for everyone else please feel free to write new steps/paragraphs/change the order.. whatever as long we move
towards the goal!
Been quite busy at work so haven't had much time to update.. but here is the latest with a few changes moving it more towards a tut from a series of point...
Code: Select all
;=============================================================================================================
INTRODUCTION
;=============================================================================================================
Pre requisites:
A firm understanding of hexadecimal, assembler, cpu operating modes like real mode, segmentation, BIOS
interrupts. Please read up on these before continuing.
A boot sector is usually the first sector of either a physical disk or partition.
The primary boot loader is 512 bytes long and MUST end with a magic signature to allow the BIOS to check
that it is bootable. The signature is:
db 055h,0aah or 0xaa55 as a short.
Floppy disks, hard-disks and any device such as CD/USB running under emulation will use 512 byte sectors.
Native CD/DVD/USB may use other sector sizes such as 2kb.
A hard-disk thats configured to multi-boot will use a slightly different structure.
Instead of one boot loader there is an MBR which will be booted first by BIOS and then usually display a menu
having scanned for bootable partitions. On selection it will load the first 512 byte sector from the partition
and switch to executing that.
Once a PC mainboard has received power it will perform the POST (Power on Self Test). If this test passes
control is passed on to the BIOS which then configures the devices present in the machine including the clock
input devices, PCI devices, emulation. It will expose a set of API functions through various interrupts and
then attempt to locate the boot-sector on the selected boot disk, load it into memory at 0x7c00 and
then transfer control to it.
----------------------------------------------------------------
What tools do I need to get a boot loader up and running?
----------------------------------------------------------------
Firstly you'll need something to code the boot-loader with. Due to the nature of the tasks a boot-loader
is required to perform you really have no option but to use assembly language. I would recommend either
fasm, nasm or tasm (if you own a legitimate copy) as these are all capable of producing a flat binary image.
fasm and tasm follow the intel syntax as opposed to nasm's at&t syntax so that is purely a matter of taste.
I would strongly recommend at this stage to use Bochs/Qemu/Virtual PC to test with as it's much easier and quicker
than rebooting a physical machine each time. Obviously once you plan to test booting CDs and USB sticks you'll
need to do it for real.
----------------------------------------------------------------
A very simple endless loop example boot code
----------------------------------------------------------------
Here is the very bare-bones skeleton of what you'll need in a boot loader. The code is written for Tasm.
As you can see we use the space after the actual code to store procedures and data as it will never be
executed here (IE: safe). We also need to ensure we pad the image file to be exactly 512 bytes in size and the
last 2 bytes are the magic signature.
.386p
code segment para public use16 'code'
assume cs:code, ds:code
org 0h
start:
jmp short $
db 510-($-start) DUP (0)
db 055h,0aah
code ends
end start
----------------------------------------------------------------
How do I create my boot image?
----------------------------------------------------------------
You'll need some tools to create the necessary image files to deploy to the various bootable devices.
The tools I use are:
partcopy to concatenate files and be able to write them into a destination file at a specific offset.
(This is great for creating the image and merging it into one of your emulators floppy or hard-disk images.
available from: http://www.virtualobjectives.com.au/utilitiesprogs/partcopy.htm
CDRTools/mkisofs to be able to create an ISO that can be burnt to CD. This allows you to specify whether to use
native or emulated bootable CD as well as how many sectors to load at boot and the load address and many other
parameters that we'll address in detail later.
available from: http://cdrecord.berlios.de/old/private/cdrecord.html
HxD Hex Editor this is a great utility for writing to any physical disk sector by sector as well as a general
purpose hex editor. You can use this to open your image file and literally copy and paste it onto either a
hard-disk or USB stick.
available from: http://mh-nexus.de/en/hxd/
Ralph Brown's Interrupt List, HelpPC and Norton Guides are also extremely helpful sources of information
to interrupts, memory addresses, bugs and basic hardware/system programming.
Search google for these.
----------------------------------------------------------------
Your first boot loader code in action
----------------------------------------------------------------
Assuming you have entered the above code the steps would be:
Assemble the file to a flat image with TASM:
c:\tasm\bin\tasm /zn /m2 /w0 boot0.asm
c:\tasm\bin\tlink /Tdc /3 boot0, boot0.bin
Copy the flat image to your emulators image file using partcopy:
c:\pcopy\partcopy boot0.bin 0 200 a.img 0
This basically means copy boot0.bin from offset 0, length 0x200 to the a.img file at offset 0.
a.img is a flat image file the size of 1.44Mb disk as created by the Bochs emulator.
To write this image to a USB stick you would now use HxD, open the boot0.bin copy all and paste-insert to
the destination drive number (be very careful doing this and make 120% sure you have the right drive otherwise
you could overwrite your hard-disks MBR!).
----------------------------------------------------------------
Objectives of the remainder of this tutorial
----------------------------------------------------------------
The majority of boot loaders out there are device-specific and require multiple boot loader sources.
Our objective is to create a single unified 2kb first stage boot loader that can be deployed to any
modern boot device and work as planned.
The first thing we need to understand and identify is what are these devices and what are the subtle
differences between them? (IE: BIOS bugs, incompatabilities or generally poorly followed standards).
The list of bootable devices we wish to support are:
- Real Floppy Disk
- Real Hard Disk
- USB Flash Stick (under FDD emulation from BIOS)
- USB Flash Stick (under HDD emulation from BIOS)
- CDROM/DVDROM (either natively, FD image emulation or HD emulation - refer to El Torito Boot Spec.)
USB boot is achieved through emulation. The BIOS will either assume the USB Stick to be a floppy or hard-disk
and emulate accordingly both in terms of what is passed into the boot-loader in the DL register as well as the
INT 13h disk functions. It is also possible that USB sticks will appear as ZIP, ZIP100, LS-120
or LS-240 drives but this is something we're not interested in supporting and can ignore.
Traditionally floppy disks required a BPB (BIOS Parameter Block) structure to be the first thing in the
512 byte boot-sector. This structure stores information about the disk's format/filesystem/os and geometry.
Some BIOSes (most new ones i've tested) are capable of booting from USB there are however several pre-requisites:
#1 the BIOS must support it and not have any bugs.
#2 the USB stick in question must be bootable (probably a USB2.0 stick - some specifically state that they're bootable)
#3 some BIOSes that boot USB under FDD emulation specifically require there to be a BPB / FAT table at the beginning
of the boot-sector/code.
For this reason I have listed it below even though in my file-system design it's completely redundant.
1) For the tutorial that we'll ignore partitions and MBR and simply load the required sectors from disk.
There are many other great tutorials on FAT16/32/MBR formats and how to load the files from within
these structures so we will not cover them.
2) There is no manadatory requirement to store the OS loaders or kernel binary within any file-system
as long as you can locate it and read it into memory.
I chose to store these linearly at the beginning of whatever device is in question and the relevant file
system information comes directly after that.
3) Because I don't use/need the BPB and my boot code is intended to be unified it will be present even in the
boot-sector of a hard-disk/cd etc.
#4 It has been reported that some BIOSes will only emulate the first 100 sectors of the USB as FDD correctly.
I haven't been able to confirm this but it would then probably be wise to ensure you have your loaders and
base USB drivers fit within that first 50kb or so.
----------------------------------------------------------------
Initial Memory map after POST (Power on Self Test)
----------------------------------------------------------------
0000:0000 - 0000:03FF [ IVT Interrupt Vector Table ]
0000:0400 - 0000:04FF [ BIOS Data Area 40h:0000 ]
0000:0500 - 0000:7B00 [ Free use memory ]
0000:7B00 - 0000:7C00 [ Boot Loader Stack ]
0000:7C00 - 0000:7DFF [ BIOS Boot Load Area ]
8000:0000 - EBDA [ if you're paranoid you shouldn't load above 8000:0000 in case the ebda is up here ]
[ i normally find EBDA to be around 0x9fc0:0000, max 4kb but 1kb here as it hits vid mem ]
0000:7E00 - 9000:FFFF [ Free use memory ]
A000:0000 - B000:FFFF [ VGA Video Memory ]
C000:0000 - F000:FFFF [ BIOS ROM Memory Area ]
FFFF:0010 - FFFF:FFFF [ Free use memory - A20 enabled ]
FEC00000 - FFFFFFFF [ motherboard BIOS, PnP NVRAM, ACPI etc ]
;=============================================================================================================
STEP 0 (BPB table for compatability with USB FDD emulated booting)
;=============================================================================================================
Here is an update to the previous simple boot code that now includes the BPB structure. The values in this
structure vary depending on the medium. These represent a 1.44Mb floppy.
org 0h
start:
;-----------------------------------------------------------------------------------
; Some BIOSes look for this BPB/BS to enable USB-FDD emulation, we'll never use it again...
; -> We'll cheat and use some of it's entries as our variables to save space.
;-----------------------------------------------------------------------------------
jmp short begin
nop
OEM DB "VXOS "
m_sectorsize DW 512 ; We're using this as our sector size variable.
Sectors_per_cluster DB 1
Reserved_sectors DW 1
Number_of_FATs DB 2
Root_entries DW 224
Total_sector_count DW 2880 ; 80*2*18
Media_descriptor DB 0F0H ; Floppy
Sectors_per_FAT DW 9
Sectors_per_track DW 18
Heads DW 2
m_cylinders DD 80
m_sectors DD 18
bsDriveNumber DB 0
Unused DB 0
bsExtBootSignature DB 29h
m_heads DD 2
bsVolumeLabel DB "VXOS "
bsFileSystem DB "FAT12 "
begin:
;=============================================================================================================
STEP 1 (Load Address Normalization)
;=============================================================================================================
Some BIOSes will load the bootsector (512bytes) to 07c0:0000 while some will load it to 0000:7c00.
In fact the BIOS may load it to any combination of seg:ofs that is equivalent to 0x7c00 linear.
The flat binary assembler source will have been created to be at a specific address (seg:ofs combination):
org 0h (assuming the segment is 07c0)
or
org 7c00h (assuming the segment is 0 and the offset is 7c00)
Either way you need to ensure this is always fixed.
In my first stage loader I work with a segment and the offset at 0.
So org 0h in the source and then a hardcoded far jump to force the segment to set:
db 0eah
dw offset cs_correction, 07c0h
cs_correction:
;=============================================================================================================
STEP 2 (Stack location choice and setup)
;=============================================================================================================
Setup the stack (I chose to keep it just below the initial load address of the boot loader).
xor ax,ax
mov es,ax ; Set ES = 0000.
mov ss,ax ; Configure Temporary Stack Just below Bootloader.
mov sp,(7c00h-16)
push cs
pop ds
;=============================================================================================================
STEP 3 (Obtain boot drive number)
;=============================================================================================================
The BIOS will pass us in the drive number of the boot drive in the DL register.
The possible values the BIOS will give us are:
00h (First floppy)
01h (Second foppy)
80h (First hard disk)
81h (Second hard disk) and so on..
Store the drive number that the BIOS passes us in DL. We must ensure DS is valid and = to CS before using
variables.
mov m_driveNo,dl
This drive number is what will be used in subsequent BIOS calls to load sectors from disk.
;=============================================================================================================
WHERE ARE WE AT?
;=============================================================================================================
Lets see now what the code looks like with these additions and while we're at it let's display a character
on screen, wait for a key-press and then reboot just to make it more interesting than sitting in an infinite loop.
.386p
code segment para public use16 'code'
assume cs:code, ds:code
org 0h
start:
;-----------------------------------------------------------------------------------
; Some BIOSes look for this BPB/BS to enable USB-FDD emulation, we'll never use it again...
; -> We'll cheat and use some of it's entries as our variables to save space.
;-----------------------------------------------------------------------------------
jmp short begin
nop
OEM DB "VXOS "
m_sectorsize DW 512 ; We're using this as our sector size variable.
Sectors_per_cluster DB 1
Reserved_sectors DW 1
Number_of_FATs DB 2
Root_entries DW 224
Total_sector_count DW 2880 ; 80*2*18
Media_descriptor DB 0F0H ; Floppy
Sectors_per_FAT DW 9
Sectors_per_track DW 18
Heads DW 2
m_cylinders DD 80
m_sectors DD 18
bsDriveNumber DB 0
Unused DB 0
bsExtBootSignature DB 29h
m_heads DD 2
bsVolumeLabel DB "VXOS "
bsFileSystem DB "FAT12 "
begin:
db 0eah
dw offset cs_correction, 07c0h
cs_correction:
xor ax,ax
mov es,ax ; Set ES = 0000.
mov ss,ax ; Configure Temporary Stack Just below Bootloader.
mov sp,(7c00h-16)
push cs
pop ds
mov m_driveNo,dl ; Store BIOS boot drive number.
xor ax,ax ; Wait for key press.
int 16h
int 19h ; Force a post reboot.
; The alternative method is to store magic value at 0040:0072h
; - 0000h - cold boot.
; - 1234h - warm boot.
;xor ax,ax
;mov es,ax
;mov word ptr es:[0072h],0000h
;db 0eah ; Jump to ffff:0000 (Alternative Reboot Scheme).
;dw 0000h
;dw 0ffffh
db 509-($-start) DUP (0)
m_driveNo db 0
db 055h,0aah
code ends
end start
This boot loader still doesn't do much, but the first important steps towards making it
reliable and compatible with multiple systems are now in place. I have chosen to keep the m_driveNo variable
right at the end pre-signature to allow us to read it using a specific offset no matter how the size of
code/data change.
;=============================================================================================================
STEP 4 (Native CDROM Booting and the Award BIOS Hack)
;=============================================================================================================
Here we can also determine if we were booted from a native CD/DVD. This can be accomplished due to the fact
that the sector size of a native/non-emulated CD is 2048 bytes (2kb). As such this entire 2kb sector would have been
loaded. If we'd created our image as 2kb and stored that to disk we'd have the normal boot signature at 512 bytes
and our own dword(32bit) signature at the end of the 2kb sector. We could read the dword at memory address
0x83FC and check for the signature and we'd know that the BIOS had loaded the full 2kb sector.
An alternative approach used in ISOLINUX is to use INT13 (Refer to Ralph Brown Int list B).
mov ax,4B01h ; Get disk emulation status
mov dl,[DriveNumber]
mov si,spec_packet
int 13
INT 13 - Bootable CD-ROM - GET STATUS
AX = 4B01h
DL = drive number
DS:SI -> empty specification packet (see #00281)
Return: CF clear if successful
CF set on error
AX = return codes
DS:SI specification packet filled
Format of Bootable CD-ROM Specification Packet:
Offset Size Description (Table 00281)
00h BYTE size of packet in bytes (13h)
01h BYTE boot media type (see #00282)
02h BYTE drive number
00h floppy image
80h bootable hard disk
81h-FFh nonbootable or no emulation
03h BYTE CD-ROM controller number
04h DWORD Logical Block Address of disk image to emulate
08h WORD device specification (see also #00282)
(IDE) bit 0: drive is slave instead of master
(SCSI) bits 7-0: LUN and PUN
bits 15-8: bus number
0Ah WORD segment of 3K buffer for caching CD-ROM reads
0Ch WORD load segment for initial boot image
if 0000h, load at segment 07C0h
0Eh WORD number of 512-byte virtual sectors to load
(only valid for AH=4Ch)
10h BYTE low byte of cylinder count (for INT 13/AH=08h)
11h BYTE sector count, high bits of cylinder count (for INT 13/AH=08h)
12h BYTE head count (for INT 13/AH=08h)
The ISOLinux boot code makes a special exception for broken Award BIOSes in which apparently the interrupt
entry for int 13h is not correct and the above function will always fail resulting in a CF set. Their approach
to fix this is as follows:
Scan the BIOS code in memory for the following code pattern:
0b8h,1,2,0bbh,0,7ch,0b9h,6,0,0bah,80h,1,09ch,09ah
which is equivalent to:
mov ax,0201h
mov bx,7c00h
mov cx,0006h
mov dx,0180h
pushf
call <direct far>
We begin the scan for this string in the BIOS ROM Memory area at f000:0000.
Assuming we're scanning via ES:DI , then ES:DI+0eh will contain the updated dword value to use as the int 13h
vector and update 0000:[13h*4] with this value. If the value is the same as the original vector
3.1
If we couldn't find the award code pattern/string we can call 4801h with all disk numbers from 80-ffh.
If one of these succeed we can check that the packets returned drive number is = to the one queried and assume
we're good to continue. If they're different check if the drive numebr we're querying is the same as the original
passed into the loader in DL and once again assume we're good to go.
Award BIOS 4.51 apparently passes garbage in sp_drive member of the above packet.
but if this was the drive number originally passed in
DL then consider it "good enough"
3.2
Intel Classic R+ computer with Adaptec 1542CP BIOS 1.02
passes garbage in sp_drive, and the drive number originally
passed in DL does not have 80h bit set.
The isolinux solution now is to OR in bit 80h, then compare to the original drive and we're good to go if they're
the same. If not then loop back to 3.1
If after all this we still have nothing we're have a particularly rubbish BIOS which may not even implement function
4b01h. In this case check that the drive numebr is AT LEAST 81h and continue, else fail the boot.
Another thing to mention at this point as implemented in ISOLINUX is the following:
Some CD-ROM BIOSes have been found to corrupt segment registers and/or disable interrupts
For this reason they don't call int 13h directly but rather via a function which saves all segment registers/flags.
;=============================================================================================================
STEP 5 (Transfer code location and jump)
;=============================================================================================================
Move the first stage loader to a safe location and continue execution.
The reason I previously selected to use 07c0:0000 instead of the offset model is that it makes this
step is a bit easier. I can copy the entire boot-loader code (512 bytes) to a new segment address
and then use a label directly after the far jump as the same offset for that jump (as the labels
offsets relative to the segment remain unchanged, this applies to variables etc as well).
I chose 0060:0000 as the destination (this is the address used by DOS 1.x apparently) and from looking
at a memory map after boot this space should be safe. This is only really necessary if you plan on using
the 1st stage loader as an MBR, but it doesn't hurt to do it anyway for our unified approach.
xor si,si ; DS:SI = begin of original bootloader (7c00:0000).
mov di,600h ; ES:DI = destination (0000:0600).
mov cx,100h ; Copy 256 words (512 bytes).
rep movsw ; Copy bootloader.
db 0eah ; Hardcode force far jmp.
dw offset bootloader_exc0,0060h ; new code addr = (0060:bootloader_exc0).
;---------------------------------------------------
; Bootloader will commence from here at new address.
;---------------------------------------------------
bootloader_exc0:
;=============================================================================================================
STEP 6 (Ensure segment registers are setup).
;=============================================================================================================
Ensure the data segment register correlates to CS after our relocation, otherwise variables won't be accessible.
push cs ; CS and DS = 0060h.
pop ds
;=============================================================================================================
STEP 7 (Determine code path based on drive number)
;=============================================================================================================
In theory, if all emulations work correctly we only need to know if the boot drive number (DL)
was >= 80h or could be masked with 80h.
That way we can use two pieces of code to load the 2nd stage loader which should support
FDD/USB emulated FDD/CDROM FDD image/DVDROM FDD image and HDD/USB emulated HDD/CD emulated HD.
;--------------------------------------------------------------------------------
; Did we get booted from a floppy, harddisk, usb or cdrom?
; -> USB will emulate a HDD boot (80h) or FDD (00h). same for CD-ROM (ElTorito).
;--------------------------------------------------------------------------------
cmp m_driveNo,80h
jae skip_not_floppy
;=============================================================================================================
STEP 8 (Preparing for loading stage 2 when using a FD or emulated FD)
;=============================================================================================================
All FD's have 2 heads, 80 cylinders and 512 byte sectors. The only value that can vary is the sectors per
track (SPT).
Based on some of the earlier linux boot code it appears that in some machines/BIOSes have issues with
with their DPT (Diskette Parameter Table) as well as not knowing the maximum SPT value for the given media.
The BIOS only knows the values for the drive and not the inserted media.
The following code can be used to kludge the SPT value in the Diskette Parameter Table
; The Interrupt Vector Table (IVT) occupies the first 1024 bytes of low memory and
; contains 256 4-byte entries. Each entry is an address to an Interrupt routine.
; The INT 1E vector (Diskette Configuration pointer) is at 0000:0078 and contains
; the address to the Diskette Parameter Table (DPT). (78h/4 = 1Eh)
;-----------------------------------------------------------------------------------
; -> Reset FDC Param Tables (apparently some BIOS have too low max sector
; count (7).. we need 36 for max 2.88 disks, this messes up multi-sector
; reads greater than the BIOS value. Too high is safe, too low not...
;-----------------------------------------------------------------------------------
push ds
mov bx,078h ; ES:BX = FDC BIOS Param Table Ptr.
lds si,es:[bx] ; DS:SI points to Param Table.
mov byte ptr ds:[si+4],36 ; Kludge Sector Per Track Count (36 = 2.88 disk).
mov byte ptr ds:[si+9],0fh ; Reset Head Bounce Time.
pop ds ; Updates INT 1E table.
mov dl,m_driveNo
xor ax,ax
int 13h ; Reset Controller for booted drive.
We then need to probe the disk by trying various read sizes to determine the SPT:
The disksizes or SPTs are: 36, 18, 15, 9
and these correspond to 2.88Mb / 1.44Mb / 1.2Mb / 720kb disks.
;--------------------------------------------------------------------------------
; Probe Disk Drive Sectors per Track.
; -> Try 36, then 18 and so on to see what works as BIOS has no definite way
; -> of knowing using INT13H, SERVICE 08h... more arcane rubbish.
;--------------------------------------------------------------------------------
mov ax,07c0h
mov es,ax
mov si,offset disksizes
probe_loop:
xor eax,eax
xor cx,cx
mov al,ds:[si]
inc si
cbw ; Extend AL to AX.
mov m_sectors,eax
cmp si,(offset disksizes)+4
jae short got_sect_size
mov cl,al
xor dx,dx
mov dl,m_driveNo
xor bx,bx ; Use future load address to overwrite (no danger).
mov ax,0201h
int 13h
jc short probe_loop ; If failed try next sector size down.
got_sect_size:
; ... more to come here
;=============================================================================================================
STEP 9 (Loading stage 2 loader for HD, emulated HD or native CD/DVD)
;=============================================================================================================
skip_not_floppy:
Step 4 needs work
Step 8 I''m still trying to determine if the DPT kludge is necessary and if the SPT probe is necessary AND if the probe may work for real FDD but cause issues with USB-FDD.
Perhaps step 8 can fall away and be combined with the work of 9 for HDD modes...
Another thought is that it might make sense to make the entire loader 2kb in size, and have the normal 512 byte signature where it must be, and ANY specialized CD-Native code etc can come after that.
Re: Boot Loader Tutorial Idea
There's a misconception in the Open Source scene that I see far too often to be comfortable with it: Criticism is contribution. I like your idea of a bootloader tutorial, and I would like to contribute the observation that not covering partitions is a severe shortcoming in my opinion.
That doesn't mean I have the time, expertise, or motivation to add it to your tutorial. But then again, I could have not contributed to it at all; would that have been better?
That doesn't mean I have the time, expertise, or motivation to add it to your tutorial. But then again, I could have not contributed to it at all; would that have been better?
Every good solution is obvious once you've found it.
Re: Boot Loader Tutorial Idea
Hello,
I think the bootloader tutorial is looking good so far. I do want to emphasize however that assembly language is not the only language that can be used in a bootloader. A lot of bootloaders use C as well for most of the bootloader. This is in reference to "have no option but to use assembly language" in the tutorial.
I do agree with Solar - you should include partitioning. If you are going to write a tutorial, write it as complete as possible
I think the bootloader tutorial is looking good so far. I do want to emphasize however that assembly language is not the only language that can be used in a bootloader. A lot of bootloaders use C as well for most of the bootloader. This is in reference to "have no option but to use assembly language" in the tutorial.
I do agree with Solar - you should include partitioning. If you are going to write a tutorial, write it as complete as possible
OS Development Series | Wiki | os | ncc
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}
Re: Boot Loader Tutorial Idea
You have to have ASM in the stage1, and the objective of the stage1 is to get to stage2 ASAP. You are also under severe space constraints for stage1 (446 bytes). I don't feel this is the environment where you'd try to link an ASM part with e.g. a C compiled part of effectively unknown size...neon wrote:I do want to emphasize however that assembly language is not the only language that can be used in a bootloader. A lot of bootloaders use C as well for most of the bootloader.
I completely agree for the stage2, though.
Every good solution is obvious once you've found it.