OSDev.org

This is quite strange. A bootloader can't get much simpler than yours! You write a character directly (without needing DS or SS:SP). And the dd-command is quite simple, too!

Maybe you use a floppy? sda isn't the floppy. I'm clueless if that isn't the reason!

Sorry, Peter

Okay, I think I might have solved the issue. I just compiled it on the target PC (I used to compile it on laptop then run it on PC).
I assumed it would be portable since the directives are so basic (mov etc is just the standard instruction set, yet it now works.)
Well, now I just have three questions remaining, can I tell nasm which platform to target? Also, I am still unsure if I need this
MBR partition record and BPB structures. Thanks!

Szustarol wrote:... can I tell nasm which platform to target? Also, I am still unsure if I need this
MBR partition record and BPB structures. Thanks!

No, you can't tell Nasm to compile for, say ARM-architecture. But the BITS command lets you specify the mode.
No you don't need BOTH MBR and BPB in one bootsector. Just the partition table at offset 446 in the MBR and the BPB in a partition or in a floppy disk.

PeterX wrote:

Szustarol wrote:... can I tell nasm which platform to target? Also, I am still unsure if I need this
MBR partition record and BPB structures. Thanks!

No, you can't tell Nasm to compile for, say ARM-architecture. But the BITS command lets you specify the mode.
No you don't need BOTH MBR and BPB in one bootsector. Just the partition table at offset 446 in the MBR and the BPB in a partition or in a floppy disk.

Shouldn't BITS 16 be portable then?

As to the second part of your post, I think I might have not understood properly, so I need MBR a partition setup in bootsector and the BPB somewhere else on
the partition?

No, the whole Assembler code is specific to one architecture. Bits specifies the target bitwidth (16=real mode,32=pmode 64=long mode). You still can't exchange 16bit ARM code and 16bit PC code, just to name an example. There was developed a kind of "high level assembler" to be exchangable. It's called C.

And a PC with Legacy BIOS (Read: Everything before (U)EFI came and except BSD) has a Master Boot Record with boot code and the partition table (parttab). The parttab lists the primary partitions. The partition begins somewhere behind the MBR and in its first sector is the bootloader code and if it's FAT filesystem it contains the BPB at offset 0x0B.

Maybe you should practice a bit of application programming before you start OS dev? I'm not bossing you around, it'S just a suggestion. Feel free to ignore it

PeterX wrote:No, the whole Assembler code is specific to one architecture. Bits specifies the target bitwidth (16=real mode,32=pmode 64=long mode). You still can't exchange 16bit ARM code and 16bit PC code, just to name an example. There was developed a kind of "high level assembler" to be exchangable. It's called C.

And a PC with Legacy BIOS (Read: Everything before (U)EFI came and except BSD) has a Master Boot Record with boot code and the partition table (parttab). The parttab lists the primary partitions. The partition begins somewhere behind the MBR and in its first sector is the bootloader code and if it's FAT filesystem it contains the BPB at offset 0x0B.

Maybe you should practice a bit of application programming before you start OS dev? I'm not bossing you around, it'S just a suggestion. Feel free to ignore it

I think I still don't understand something. You say that MBR has boot code, and then that behind MBR there is a bootloader. While reading about this on wikipedia they say that that MBR contains the bootloader. I think I got it all mixed up, I thought the first sector of the drive IS the MBR and it contains the partition table as well.

The MBR (the first sector of the hard disk) contains both the boot code AND the partition table. The first sector of the partition (that's not the first sector of the disk) contains bootcode, too, sometimes. The MBR loads something and this "something" is on some systems the partition's first sector. The partition's first sector contains code AND BPB.

Well, this all is quite fundamental.

It would be interesting to compare the binary outputs on the two systems to find the differences and try to figure out their cause.

The USB usually can be treated by BIOS in two ways: as a floppy, and as a hard disk. In either case, it should be formatted accordingly.

Hard disks have MBR in the first sector. It starts with some code, and contains a partition table in the end. The code is expected to chain load the first sector of an active partition and pass control to it. Each partition can have different filesystem, and the format of fisrt sectors of the partitions depends on the filesystem used.

Floppy disks can be viewed as a single partition. They are usually formatted as FAT12, so the first sector starts with a short JMP with NOP or a near JMP, then 8-byte identification string (like 'MSDOS5.0'), then the BPB, then the bootloader code. (If you want to force the null CS, then you may do the far jmp after the BPB.)

Either way, BIOS should ideally just load the fisrt sector of the medium to physical address 0x7C00 and pass control to it. But some manufacturers perform validity checks and refuse to load medium of unknown format. Therefore, some BIOSes expect BPB, may check for specfic identification string, may even overwrite the BPB, most commonly, geometry information. Some BIOSes expect partition table. The osdev wiki article you referred to mentions that you can't be sure how BIOS treats the USB stick, so you should be prepared for both scenarios, that's why it recommends to build such strange first sector containing everything BIOS might want to find.

And one more note: The dd-command you use destroys the parttion table. So you may loose all of your data if you write to a real harddisk that way.

I use these commands in my makefile:

A VBR must start with a near or short jump, or it may be considered invalid. If you use a short jump, it must be followed by a NOP.
A VBR must contain an OEM name, or it may be considered invalid.
In a VBR, this field must be either a value greater than 0xFFFF or 0. Since you're specifying a volume with 2880 sectors, use 0.
The initial stack prepared by the BIOS may be very small. It may even be too small to call certain BIOS functions. Try setting up the stack before calling this function. Be careful not to let any interrupts happen between setting SS and SP. (Instructions that set SS automatically prevent any interrupts until after the following instruction.)
A partition must not start in LBA 0 (CHS 0/0/1) or the MBR may be considered invalid. Your translation between LBA and CHS doesn't appear to be correct, either, which may also cause the MBR to be considered invalid. The typical geometry uses 63 sectors per track and 255 (not 256) heads per cylinder.

Personally, I wouldn't recommend trying to combine a BPB and partition table into one sector. However, I'm not yet certain what you must do to prevent the BIOS from detecting a BPB or partition table that doesn't actually exist.

Szustarol wrote:

PeterX wrote:I might be wrong, but isn't a far JMP at the beginning of a FAT floppy MBR wrong?

I'm doing it to ensure that I end up at segment 0x0000, is there other way to ensure this? (I might be at 0x7c00:0x0000)
Also I've seen many people do this before so that's why I did it. Of course I'm not trying to say You're wrong Sir, just
curious why there are so many tutorials showing this procedure

Nope, he is right. The boot record *should* start by three bytes, a SHORT jump plus a NOP. And the reason why so many tutorials showing this is because there are software that checks for this short jump (as a sort of additional magic to 55AAh, the first byte should be EBh, the second byte must be less than 80h and the third byte should be 90h). This is not written anywhere, but I had issues too with certain partitioning software if the boot sector didn't started like that. (Just for the records, the FAT code MUST jump over the BPB area, and in theory an E9h jump could be used as well).

Szustarol wrote:Also, this site https://wiki.osdev.org/Problems_Booting_From_USB_Flash suggest's that MBR partition table is required for a proper boot,
but You guys say that I is only needed when booting multiple partitions. Can I ask for some clarification?

Again, this is not a must, but certain firmware MIGHT check for an MBR table. It is better to have it, it won't hurt on floppies either (not much sense there, just create one partition that covers the whole diskette). Same with USB flash drives.

Szustarol wrote:Also I have seen people saying that BPB is needed (or some BIOSes might not boot), or even that some BIOSes might
overwrite this area so It's secure to have data, not code there. How much truth does this hold?

Please note that from the BIOS' perspective, there should be absolutely NO difference between MBR (without BPB) and VBR (with BPB), all BIOS cares about is loading the first sector and check if it ends in a 55AAh. It should not matter if that first sector is loaded from a diskette or a disk, or that the loaded sector is an MBR or VBR, the procedure is the same according to the BIOS Boot Specification. Some buggy BIOS although does not comply with this, and do further checks, which is just plain wrong.

Regardless BIOS doesn't care about BPB. It is named not because it's a BIOS area, but because it contains information needed to call BIOS functions. I find it highly unlikely that any BIOS would check for that, as there's not much constant bytes in a BPB that could be used as magic, and MBR as well as non-FAT filesystems doesn't have BPB anyway.

Here's the original MBR code's source (no BPB).
And the original FAT VBR's source (with BPB and the EBh+90h magic).
Win 98 VBR's source (with BPB and the EBh+90h magic).
And Win 8/10 NTFS VBR's source (with a fake (non-FAT) BPB and the EBh+90h magic).

Cheers,
bzt

bzt wrote:Again, this is not a must, but certain firmware MIGHT check for an MBR table.

When booting from a USB flash drive, most firmware attempts to verify either the partition table or BPB (sometimes both) in order to determine whether the flash drive should be emulated as a hard drive or a floppy drive. Try it for yourself: on most computers, the same USB flash drive will appear differently in the BIOS boot menu depending on whether it contains a valid partition table or a valid BPB. One Dell PC I own lists it as a hard disk when there's a valid partition table, and a Zip disk when there's a valid BPB.

Note that "valid" basically means "created by Windows" since different firmware will use different methods to decide what's valid.

bzt wrote:Some buggy BIOS although does not comply with this, and do further checks, which is just plain wrong.

I don't see how the BIOS can choose the emulation type otherwise.

bzt wrote:Regardless BIOS doesn't care about BPB.

When booting from a USB flash drive in floppy disk emulation mode, many BIOSes will adjust the emulated disk geometry to match the BPB. Some will adjust the BPB to match the emulated disk geometry (and cause problems for you if you've put something else where the BPB goes).

The Dell PC I mentioned earlier doesn't do a very good job of validating the BPB. It will accept a BPB that indicates zero heads per track, which causes it to hang during POST when it tries to set up the emulated geometry.

Perhaps a bit of a history lesson might clarify things. Maybe not, in which case you can ignore this. Also, if anyone notices an error in this, please feel free to correct me. Note that much of this is no longer relevant at all, as most systems today - including all Ryzen systems I know of - have UEFI instead of Legacy BIOS, and the way those handle booting is quite different if you haven't set them to a Legacy mode.

TL;DR: don't change the MBR. Chances are a modern hard disk won't actually use it anyway (it will be there, as a 'protective MBR', but the actual drive information would be in a GUID Partition Table and an EFI System Partition instead, and the MBR is only checked to make sure it isn't a legacy drive), and even if you do, you don't have any reason to write to the drive's zero sector. You need to find out what the zero sector of the partition you want to boot from is, and write to that instead. You'll need to write a special-purpose utility to do that, rather than using dd or similar tools. Also, your motherboard may support Legacy BIOS routines (using the CSM), but newer ones - including most Socket AM4 mobos, which is what you'd have for a Ryzen CPU - often don't, so using INT 0x10 probably won't work anyway.

When IBM first designed the PC, they had some faulty expectations about the product - through no fault of their own, their crystal ball was out of commission at the time and they'd run out of tea leaves - four of which were a) that CP/M would be the dominant disk-based OS, with UCSD p-System being for top-tier academic use and that PC-DOS thing being the low-cost option (and for their own part, Microsoft really would have preferred Xenix to be their main OS product, and saw PC-DOS as a stopgap in the same way Intel saw the 8086 as a way to carry them over while they developed the iAPX 432); b) the majority of customers would buy the cassette tape based model, since the floppy drive option was half again the price of the intended base model, so the cost of the floppy OS didn't matter; c) the product life would be short, since people would soon get over this 'microcomputer' fad and go back to using batch-processed mainframes as Dog intended; and d) no one would be daft enough to connect an expensive hard drive to a home computer, and even if they did, CP/M would handle it just fine.

Most of these would have been reasonable assumptions in 1980, when the project was started. The first two were already starting to collapse by the time the PC went on sale, however, as users moved away from cassettes, and CP/M-86 got delayed due to unexpected problems unrelated to the PC project.

Just to put this in perspective, in 1980 when the PC was being designed, the price of a 5.25" form factor 5MiB Shugart fixed disk hard drive (as opposed to the massive disk-pack based beasts which IBMers saw as real hard drives) was somewhere around $1500 in then-current dollars, which is about $4700 today. The base unit price of a 16KiB cassette model PC was projected at $1500 (1980), and the 64KiB floppy drive version at $2000. After factoring in the additional controller hardware, a hard drive would have roughly doubled the price of the most expensive model.

Now, it should be noted here that the original plan was for an 8-bit machine, similar to the Apple II or Commodore PET (the VIC-20 wasn't out just yet, and the C64 was a few years in the future), with the first idea being simply to buy out Atari's home computer division. While legend has it that Bill Gates talked them out of this, they hadn't even talked to him about the PC yet; the actual decision to go to a 16-bit system had to do with internal IBM politics. But that's irrelevant right now.

The point is, neither they nor Microsoft thought that PC-DOS would ever need to handle more than one type of disk. The 360KiB 5.25" floppy was good enough for anyone, at least in the four or five years PCs were expected to be used before a completely different system replaced them (while IBM was all about backwards compatibility usually, at the time the small computer market was shifting so rapidly that everyone saw it as a given that upgrading would mean moving to a completely new hardware and software environment. For the most part this was right, too - companies like Apple, Tandy, and Commodore would be shifting focus to entirely different hardware platforms by 1983, and while they still supported and sold their older systems, the break going from the Apple IIe, TRS-80, and C64 to the Lisa and Macintosh, the CoCo, and the Amiga was seen as natural and necessary.

It was true for the PC eventually, too, but the transition was smoothed out by the fact that while the AT and PS/2 hardware were radically different from the 5150 (and each other) in some ways, they all could still run PC-DOS software despite both IBM and Microsoft already wanting to shift way from that, too.

The sum of this was that the original PC BIOS always booted from the first sector of the floppy, and didn't need any other information from the disk itself to do so. The first versions of PC-DOS (and MS-DOS) had no BPB, since they knew what the disk geometry was going to be. I'm not even sure if the boot signature was a thing yet or not, frankly, though I imagine it was - while most early PCs had only one drive, some had two, and in either case swapping disks was a common procedure when running larger programs (less so than with system which only had single-head drives, like the Apple II, but still), so many disks didn't include a boot sector at all, as you didn't want to waste a usable sector if you didn't need to.

It was only around 1983, when hard drive prices had dropped significantly, 3.5" floppy disks and high density and 5.25" drives were starting to come out, and CP/M-86 had bungled its intended role as the main PC disk OS, that Microsoft reconsidered this a bit. The fact that - due to some clever wording in their contract - they were free to sell MS-DOS separately, and could even sell as an OS for unrelated 8086/88 hardware such as the AT&T 6300 and the DREC Painblow - sorry, I mean 'DEC Rainbow' - also caused them to reconsider DOS design. They needed a way to determine the disk geometry at boot time, without relying on any changes to the ROM BIOS itself - it was long before flashable BIOS, so changing the BIOS on an existing system would require a soldering iron. They settled on putting the BIOS Parameter Block at the start of the boot sector, but since the BIOS always jumped to the start of the loaded sector, they had to have a jump at the very beginning to go around the BPB. This, and the hard drive support I'll discuss next, appeared in PC-DOS 2.0.

When clone PCs started coming out (as opposed to work-alikes such as the Rainbow, which behaved almost the same but weren't actually compatible), some of the clone BIOS designers started treating that initial JMP SHORT as a sort of secondary boot signature; this was useful, since at the time PC-DOS/MS-DOS was very nearly the only game in town. Using that served as a secondary check, since it was possible, however unlikely, that a data disk would just happen to have a 0x55AA at the end of the sector zero. While not all BIOSes required this, enough did that it became standard practice, so other PC OSes at the time - which mostly meant DR-DOS and 4DOS, both of which used FAT12 anyway and thus had a BPB on their disks - had to follow Microsoft's lead on this.

Meanwhile, by 1983 IBM had introduced the PC/XT, which still used the 8088 but now had more on-motherboard memory capacity and had a 10MiB hard drive in the top-tier models. While they had added hard drive support to the BIOS, it still would always boot from the first sector. Both IBM and Microsoft could see that this could lead to problems, however, as it was common practice even then to partition large drives into several smaller logical 'drives'. Thus, a developer at MS designed the original Master Boot Record to accommodate multiple drive partitions. These were introduced with PC-DOS 2.0 as well.

Basically, the MBR was a compromise between the rather simple boot process IBM used, and the need to support multiple partitions. It let the BIOS load in sector zero, just as before, but allowed the system to have a volume record inside that same sector both to keep track of the partitions themselves, and to let the owner configure which of those partitions would be the boot partition. This was fortunate, since it also meant that having a boot manager for multi-booting would be feasible later on, but that wasn't the intended purpose originally. It also meant that reformatting a partition didn't require the MBR bootstrap code to be changed, only the partition table data.

This is important to understand: you, as an OS developer, should never touch the MBR of the disk when installing your volume boot code. You may need to write a utility that changes the partition data at some point, but that's separate from the process of installing your volume boot record. There is hardly ever a need to touch the MBR boot code itself, so writing your VBR to the zeroth sector of the disk itself is the Wrong Thing To Do.

In the original design, the MBR would be loaded by the BIOS to 0000:7C00, just as a floppy disk's boot sector would be. The original MBR design has two sections, a minimal boot loader in the first 446 bytes, and the volume record table in the remaining 66 bytes. While later variants on the MBR would have different overall structures, the one thing that remained constant was the size and location of the partition table.

The way it worked was simple: the MBR bootstrap code would first relocate itself to some other part of memory. It then goes through the partition table to find which one was the boot volume, and then load the first sector of the boot volume - the Volume Boot Record, which is the part you'd be adding - into 000:7C00 (this was meant to minimize the differences between a floppy boot sector and a VBR).

As with the JMP SHORT at the start of the boot record, various BIOS implementations started using the partition table header as a 'validity signature'. This makes sense, since a hard drive without an MBR is pretty much unreadable, and will need to have the MBR replaced (or at least the partition table, if there are not bootable partitions). Thus, even though it wasn't in any official specification originally, it became a requirement for any hard drive to have a valid partition table.

The partition table also includes information about the drive geometry and the type of file system the partition is formatted with, so any tool for formatting a drive will also need to be able to set the volume type in the partition table.

With USB drives, as others have already said, there are to ways to handle the disk: it can either emulate a floppy, or it can emulate a hard disk. I don't know off-hand how this is handled, but I will say that a) emulating a floppy limits the total usable size of the drive, and b) IIUC, most newer Legacy BIOS and UEFI BIOS have dropped support for floppy emulation.

Schol-R-LEA, you wrote:
1.) Never touch MBR directly
2.) Never use dd (EDIT: Never use dd in this context)
3.) Don't use MBR but instead VBR (partition boot sector)
4.) OS developer should never write code to/for MBR
5.) Use special tools

I disagree with 4.)
Are you sure about all of them? And do you mean all of them applied to UEFI PCs (=not to Legacy BIOS PCs)?

And what tools for (5.) would that be?

Happy hacking
Peter

PeterX wrote:Schol-R-LEA, you wrote:
1.) Never touch MBR directly
2.) Never use dd (EDIT: Never use dd in this context)
3.) Don't use MBR but instead VBR (partition boot sector)
4.) OS developer should never write code to/for MBR
5.) Use special tools

I disagree with 4.)
Are you sure about all of them? And do you mean all of them applied to UEFI PCs (=not to Legacy BIOS PCs)?

I was mostly speaking of Legacy BIOS; the UEFI setup is entirely different, anyway, so the MBR isn't really relevant. For a drive set up with a GPT, most UEFI implementations treat the Protective MBR solely as a signature.

As for not writing code for the MBR, there are a handful of instances where one might want to change the MBR code, but honestly, they are few and far between. I would say that, unless you are either writing a repair tool to fix the MBR after it was corrupted or infected, or doing something involving hardware customization with a non-standard drive type, then 99.999% of the times when you might think, "I should change the MBR boot code for this", you are mistaken.

Or to put it another way: partition managers such as fdisk or parted must change the partition table, as do most OS or boot loader installers, but they won't alter the MBR code if it already exists, unless they detect corruption.